Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance

ABSTRACT

An imaging system includes an illumination device for illuminating a target. A surgical microscope receives light from the target, the surgical microscope comprising at least one optical output port at which at least a portion of the received light is provided as an output from the surgical microscope. A tunable filter receives the portion of the received light provided as the output from the surgical microscope, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter. A high-resolution, broad-bandwidth electronic camera receives the light of a plurality of wavelengths selected by the tunable filter, the electronic camera converting the light of a plurality of wavelengths selected by the tunable filter to a plurality of electrical signals. A processor processes the plurality of electrical signals to form an image of the target.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 14/373,443 with a § 371(c) date of Jul. 21, 2014, which is a 35U.S.C. § 371 filing of International Application No. PCT/US2013/022266,filed Jan. 18, 2013, which claims priority to U.S. Provisional PatentApplication 61/588,708 filed Jan. 20, 2012, the content of each of whichis incorporated herein by reference.

GOVERNMENT RIGHTS

This invention was made with government support under grant numberR01NS052274-01A2 awarded by the National Institutes of Health. Thegovernment has certain rights in the invention.

FIELD

The present application relates to the fields of surgery and imaging ofsurgical sites. In particular, the present application relates tohyperspectral fluorescence and reflectance imaging to improve ability ofa surgeon to distinguish tissue types during surgical procedures, suchas, for example, surgical procedures involving tumor resection in thebrain or other organs.

BACKGROUND

There are many types of lesions treatable with surgical removal ormodification. These lesions include tissues abnormal for any location inthe body, such as malignant (or cancerous) tumors, and manyslower-growing “benign” tumors. These lesions also include tissues thatare abnormal for their location in a particular organ, but resemblenormal tissues found in other locations in the body. Other lesions mayincorporate material foreign to the body, including bacteria, viruses,or parasites, and associated zones of immune reactions. Still othersinvolve developmental anomalies, such as arteriovenous malformations andberry aneurisms. Other lesions may incorporate scars and adhesions fromprior illness or injury. While lesions are of many kinds, it isgenerally desirable for a surgeon to be able to visualize the lesionbeing treated and to be able to discriminate between normal and lesiontissues.

Many tumors and other lesions do not have a capsule or other connectivetissue that separates them from nearby normal tissues; they may haveirregular boundaries. Invasive malignant tumors in particular often haveinfiltrations and filaments containing malignant cells that penetrateinto adjacent normal tissue. Some benign and some malignant tumors havetissue that superficially resembles tissue like that from which thetumor arose in both color and, to a certain extent, in texture. Sometumor types, including gliomas, have motile cells that may migrate ashort distance away from the tumor into normal tissue; once these cellshave found a hospitable location they may grow and form a new spinofftumor. The new tumor may or may not become attached to the parent tumor,if it becomes attached it may resemble a filament of tumor. Either way,the tumor may develop a somewhat ragged edge with filaments and spotspenetrating into adjacent tissue.

To reduce recurrence of many tumors after surgical treatment, includingmany malignancies, it is considered desirable to remove all detectableportions of the tumor.

While filaments of tumor, and motile cells, may stop extending for atime when they reach an organ capsule, resulting in tumor encapsulatedin the organ, it is often undesirable to remove an entire organ or organlobe—especially when an organ is critical for life and the tumor may nothave invaded the entire organ. For example, removal of more brain tissueor spinal cord than necessary can cause life-altering neurologicalimpairment. Similarly, it may be desirable to save as much as possibleof a patient's only kidney. There are other organs and body structureswhere tumors may form but where it may be desirable to retain as muchpost-surgery organ structure and function as possible.

Invasive filaments and clones from formerly motile cell portions oftumors may not be readily visible to a surgeon—even under magnification.Other lesion types may also have portions that have color and structurethat resemble nearby healthy tissue, making it difficult for the surgeonto distinguish the lesions from the healthy tissue.

A prior method of ensuring complete tumor removal while retaining asmuch organ as possible involves a pathologist cooperating with thesurgeon. The surgeon removes the tumor and some adjacent tissue, whilethe pathologist immediately examines frozen sections to verify that theremoved tissue includes a tumor-free margin. Should tumor portions befound to extend to boundaries of the removed tissue, extension of tumorbeyond the removed tissue is assumed and more adjacent tissue is removedbefore closing the incision. This method tends to be slow, requiringextended anesthesia times and repeated frozen sections, and may requireremoval of more tissue than necessary because frozen sections can onlybe performed on tissue after the tissue is removed from the patient.Further, not all abnormal tissue types are readily distinguished in afrozen section. An alternative or supplemental method involvespathological examination of stained sections to verify complete tumorremoval with removal of adequate margins of healthy tissue, howeverstained sections often take so much time to prepare that any furtherremoval requires re-operation.

It is desirable to find improved ways of locating and identifying duringsurgery abnormal tissue, including tissue both abnormal for the organand malignant tissue, including small invasive branches of tumors intissue adjacent to tumors.

Generally, surgeons treat lesions that are visible to them duringsurgery. At times, lesions and tumors may lie under the surface of anorgan, or under a visible and exposed surface of an operative site,where they may be obscured by overlying tissue and not readily visible,or may have poor contrast relative to surrounding stroma. It isdesirable to make these lesions, including portions of malignant tumors,visible to a surgeon so that they can be more readily treated, with lessnormal overlying tissue damaged during treatment, than with currenttechniques.

It is known that some fluorescent compounds will accumulate in tumorsand other abnormal tissues. Further, it is known that some prodrugs,such as 5-aminolevulinic acid (5-ALA) can be metabolized intofluorescent compounds to a greater extent in some tumor tissues than insurrounding normal stroma. Marking of tumors with 5-ALA metabolites andusing resultant fluorescence at the surface of an operative site toguide surgery has been reported in the literature. For example Stummer,et al., Fluorescence-guided surgery with 5-aminolevulinic acid forresection of malignant glioma: a randomized controlled multicentre phaseIII trial, Lancet Oncology, Lancet Oncology, Lancet Oncol., 2006. 7(5):p. 392-401, published online Apr. 13, 2006 at oncology.thelancet.com,reports that removal of malignant glioma tumor tissue marked withfluorescent metabolites of 5-ALA and fluorescing in the visible spectrumat the surface of an operative site under violet-blue excitation lightduring surgical treatment of glioma improved extent of tumor resectionand enhanced six month progression free survival in human subjects.Similar studies have also been performed in mice. It is expected thatthese results may apply for other lesion types.

Experiments have been previously conducted with tomographic fluorescentimaging of concentrations of fluorescent compounds, or fluorophores, inbiological tissues. Vasilis Ntziachristos and Ralph Weissleder,Charge-coupled-device based scanner for tomography of fluorescentnear-infrared probes in turbid media, Medical Physics, Vol. 29, No. 5,May 2002, have reported a use of diffuse optical tomography in “smallanimal geometries”. The device of Ntziachristos and Weissleder, however,operates in a transmission mode. In transmission mode, light istransmitted into a turbid medium, and emitted light is detected fromseveral points on the surface, including points on an opposite side ofthe turbid medium from the points where light is applied. The turbidmedium of Ntziachristos and Weissleder is about one inch thick, farthinner than many human organs and tissues, because it is used in smallanimals such as laboratory mice. The device of Ntziachristos andWeissleder applies light to the medium from a pulsed laser, and detectslight from the medium, through an arrangement of optical fibers placedabout the medium. The device of Ntziachristos and Weissleder uses anintensified charge-coupled device (ICCD) camera to time-resolve thedetected fluorescence in a time-domain system. Additional devices foroptical imaging of biological tissues have been reported in Hillman, E.,Optical brain imaging in vivo: techniques and applications from animalto man. Journal of Biomedical Optics, 2007. 12(5): p. 051402.

Frederic Leblond, et al, Diffuse optical fluorescence tomography usingtime-resolved data acquired in transmission, in Multimodal BiomedicalImaging II, vol. 6431. Proceedings of the International Society ofOptical Imaging (2007) disclosed a time-dependent method for solving thediffusion equation (DE) for light propagation in tissues, andreconstruction algorithms for use therewith.

US Patent application 20080218727, to Djeziri, et al., entitled MethodAnd Apparatus For Optical Image Reconstruction Using ContourDetermination, 2008, describes the importance of determining tissuecontours and the impact of tissue contour in diffuse optical tomographyreconstruction algorithms in context of intact breast imaging. Djeziriproposes raster-scanning to determine an intensity profile, and usingthe intensity profile as a surface contour of the breast. He specifiesusing an optical fluid to fill space between the breast surface and theoptical fibers of his diffuse optical tomography apparatus duringdiffuse optical imaging.

During surgery, use of an optical fluid to fill space between transmitand receive optical fibers is often difficult because this fluid wouldneed to fill the surgical wound, could infiltrate into the patient, andmay require a dam around the wound. Further, operation of a diffuseoptical imager in transmission mode may prove difficult if the body partbeing operated upon is thicker than an inch—as are the brain, kidneys,and many other organs.

Most tissues of the human body are soft tissues; these tissues areinherently flexible and readily deformable. Further, many of these softtissues interface with other tissues along boundaries where considerablemovement may take place. During surgery, as adjacent structures such asskin, muscle, and bone are moved and pressure applied to soft tissueswith instruments such as retractors, these tissues will deform andshift. Since these tissues may deform readily both between imaging andsurgery, and during surgery, it is common for surgeons to find lesions,including tumors and foreign objects, and other surgical targets are nolonger in the exact positions they occupied in preoperative images.

For a surgeon to properly treat these lesions, the surgeon must locatethem during surgery. Further, for surgeons to avoid unintended damage toother nearby structures, it may also be necessary to locate particularportions of those other structures precisely during the surgery.

MRI and CT imaging are often used to provide high resolutionpreoperative images of surgical targets. The equipment required to makethese images is bulky, expensive, and not easily incorporated into anoperating-room environment. Further, the intense magnetic fieldsrequired for MRI may be incompatible with other operating roominstruments and equipment, and radiation emitted by CT machines mayrequire surgeon and staff wear bulky and heavy lead-lined garments orleave the room during intraoperative imaging.

In Hartov, et al., Error Analysis for a Free-Hand Three DimensionalUltrasound System for Neuronavigation, Neurosurgical Focus 6 (3), 5 Aug.1999, it was suggested that sensors, such as those produced by AscensionTechnology Corporation, Milton, Vt., be used to track a handheldultrasound transducer in three dimensions. An alternative system uses aStealthstation® 3-D surgical navigation system produced by Medtronic, ofMinneapolis, Minn., for tracking instruments in three dimensionsrelative to a patient during surgery.

There are also chromophores naturally present in biological tissues,including human tissue. A leading such chromophore is theiron-containing heme group—as found in myoglobin and hemoglobin. Heme isgenerally found in both oxygenated and de-oxygenated forms in the body,and it is well known that absorption spectra of heme differs between theoxygenated and de-oxygenated forms; this difference in absorption may beused to identify tissues having different oxygen concentrations.

Many malignant tumor types have high metabolic activity due to rapidcell division and growth. These tumors often outgrow the local oxygensupply; some tumors stimulate rapid proliferation of blood vessels toovercome this, and some tumors develop core areas of low oxygen tensionand may develop necrotic portions. Imaging of heme concentrations andoxygenation may assist in locating some types of malignant tumor tissue,as well as of imaging tissues such as muscle, bone marrow, liver,spleen, and blood vessels including arteriovenous malformations andaneurysms that naturally have high heme concentrations. Djeziri'sdiffuse optical imaging system of the breast described above is intendedto visualize heme concentrations, such as those that result from rapidblood-vessel proliferation.

Muscle, including cardiac muscle, and brain activities are known toconsume oxygen. A normal physiological response to this increase ofoxygen consumption with activity is to dilate blood vessels to increaseblood flow in affected tissue. In many diseases, including peripheralvascular disease, and cardiovascular disease, as well as cerebrovasculardisease, ischemic bowel disease, the centers of some types of tumors,and other conditions, this physiological increase of flow is impairedresulting in a local decrease in oxygenation of heme. A significantdecrease in oxygenation may produce pain or other signs and symptoms, asin intermittent claudication or angina. Further, mapping increases inblood flow due to brain activity can be of interest in monitoringactivity in the brain.

For all these reasons, it is desirable to be able to map areas of hemeconcentration, to map areas of oxygenated heme and de-oxygenated heme,and to be able to view dynamic changes in oxygenation with tissueactivity.

Other chromophores naturally present in some tissues, including sometypes of tumor tissues, are naturally fluorescent.

Swartling, et al. Fluorescence spectra provide information on the depthof fluorescent lesions in tissue, Optics Letters, 2005, 44(10) pp1934-1941 found that emissions from fluorophores have spectra thatdepend on the depth of the fluorophores.

The extent of resection for brain tumor procedures has been shown tocorrelate with patient survival and quality of life. Accurate tumortissue identification with a surgical microscope alone can bechallenging because of lack of visual or mechanical features in tissueto discriminate between normal tissue and tumor. Fluorescence-guidedneurosurgery marks tumor tissue with a fluorescent contrast agent, anduses fluorescence detection technologies to identify tumor tissue usingthe fluorescence signals emitted from tumor tissue. Current surgicalmicroscopes enabled for fluorescence imaging typically performsingle-band, single-spectral-wavelength, detection. Although useful,significant levels of tumor tissue can be left undetected using thisfluorescence imaging approach.

Prior fluorescence detection technologies in the operating room displayfeatures and functionalities such as: i) surgical microscopes modifiedfor broad-beam fluorescence imaging currently allow for wide-field, forexample, up to 50 cm², single-band, 620-720 nm, non-spectrally resolvedfluorescence detection that assesses fluorescence qualitatively, withoutaccounting for non-linear effects of tissue optical properties on theemitted fluorescence; and ii) surgical point-probe spectroscopy devicesfor fluorescence detection currently allow single-point, for example a 1mm², spectrally-resolved fluorescence detection that may or may notmeasure fluorescence quantitatively.

SUMMARY

According to a first aspect, an imaging system is provided. The imagingsystem includes an illumination device for illuminating a target and asurgical microscope for receiving light from the target. The surgicalmicroscope includes at least one optical output port at which at least aportion of the received light is provided as an output from the surgicalmicroscope. A tunable filter receives the portion of the received lightprovided as the output from the surgical microscope, the tunable filterbeing tunable to pass a filtered portion of the received light, thefiltered portion of the received light having a plurality of wavelengthsselected by the tunable filter and provided as output from the tunablefilter. A high-resolution, broad-bandwidth electronic camera receivesthe light of a plurality of wavelengths selected by the tunable filter,the electronic camera converting the light of a plurality of wavelengthsselected by the tunable filter to a plurality of electrical signals. Aprocessor processes the plurality of electrical signals to form an imageof the target.

According to another aspect, a surgical microscope system is provided.The surgical microscope includes an illumination device for illuminatinga target and at least one optical output port at which at least aportion of received light received from the target is provided as anoutput. A tunable filter receives the portion of the received lightprovided as the output from the optical output port, the tunable filterbeing tunable to pass a filtered portion of the received light, thefiltered portion of the received light having a plurality of wavelengthsselected by the tunable filter and provided as output from the tunablefilter. A high-resolution, broad-bandwidth electronic camera receivesthe light of a plurality of wavelengths selected by the tunable filter,the electronic camera converting the light of a plurality of wavelengthsselected by the tunable filter to a plurality of electrical signals. Aprocessor processes the plurality of electrical signals to form an imageof the target.

A real-time hyperspectral fluorescence and reflectance imaging device isprovided that may, in an embodiment, be integrated into a surgicalmicroscope. In other embodiments, the device is used with anotheroperating room instrument or used independently. The device provides:

-   -   i) reflectance hyperspectral imaging to quantify        reflectance-derived optical parameters, including but not        limited to, oxygen saturation, oxy- and deoxy-hemoglobin,        myoglobin, and bilirubin concentrations, and spectral        distortions;    -   ii) reflectance hyperspectral imaging to provide information        about the tissue and to provide information necessary to correct        the fluorescence images for light absorption and scattering        effects; and    -   iii) quantitative hyperspectral fluorescence imaging for        intraoperative guidance of neurosurgical procedures, including        but not limited to brain tumor resection procedures to more        accurately and quantitatively detect tumor tissue across the        whole field of view using spectrally resolved fluorescence        imaging.

In another embodiment, a method of imaging includes: providing a whitelight to a target; taking a series of images through a tunable opticalfilter of the target while tuning the filter to a plurality ofwavelengths, the series of images forming a spectral image; providing astimulus light to the target; taking a fluorescence emission image ofthe target; processing the spectral image to determine an absorption anda scattering parameter at pixels of the fluorescence emission image; andcorrecting the fluorescence emission image using the absorption andscattering parameters to produce a corrected fluorescence emissionimage.

In another embodiment, a method of imaging includes providing a whitelight to a target; taking a series of images through a tunable opticalfilter of the target while tuning the filter to a plurality ofwavelengths, the series of images forming a spectral image; processingthe spectral image to determine an absorption and a scattering parameterat pixels of the images; and decomposing the spectral image to provideimages of components of the target, the components including specificscattering and absorbing substances expected to be in the target.

Therefore, advantages of the present system and method include theability to provide: i) wide-field and ii) spectrally-resolvedfluorescence detection for iii) quantitative fluorescence andreflectance assessments in near real time. The present system combinesthe advantages of current wide-field fluorescence imaging approaches,which allow for a wide field of view of the surgical field, and thespectrally-resolved and quantitative capabilities of single-pointspectroscopy devices, to provide the surgeon with quantitativefluorescence and reflectance images of the whole field of view.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features and advantages of the invention will beapparent from the more particular description of preferred aspects ofthe invention, as illustrated in the accompanying drawings in which likereference characters refer to the same parts throughout the differentviews. The drawings are not necessarily to scale, emphasis instead beingplaced upon illustrating the principles of the invention.

FIG. 1 contains a schematic diagram of a through-microscope spectralresolved quantitative fluorescence imaging system and a surgicalmicroscope enabled for fluorescence imaging, according to someembodiments.

FIG. 2 contains a schematic functional block diagram of a system,according to some embodiments.

FIG. 3 contains a schematic functional block diagram of workflow of thequantitative fluorescence imaging according to some embodiments.

FIG. 4 is a schematic functional block diagram of the intraoperativestandard probe system according to some embodiments.

FIG. 5 illustrates the PpIX fluorescence spectrum (left) and the diffusereflectance spectrum (right) measured in a high-grade glioma followingtissue excitation with blue and white light, respectively.

FIG. 6A-6F illustrate raw and corrected emission spectra measured fortissue phantoms, according to some embodiments. Three different PpIXconcentrations and a range of absorption and scattering values wereused: (FIG. 6 a , FIG. 6 b , FIG. 6 c ) varying absorption with constantscattering; (FIG. 6 d , FIG. 6 e , FIG. 60 varying scattering withconstant absorption.

FIG. 7 illustrates quantification of PpIX concentration in phantomsbased on raw fluorescence intensity (X=635 nm), according to someembodiments.

FIG. 8 illustrates quantification of PpIX concentration in phantomsbased on (a) raw fluorescence intensity (λ=635 nm), and (b) estimatedC_(PpIX) levels derived following correction of the raw fluorescence,according to some embodiments.

FIG. 9 illustrates (a) empirical determination of the optimal α valuefor PpIX quantification and (b) relationship between the estimated anactual PpIX concentration in phantoms following correction of the rawfluorescence, according to some embodiments.

FIG. 10 contains ROC curves quantifying the performance of threedifferent fluorescence measurement schemes for optical datasets acquiredin vivo during 14 brain tumor resection procedures.

FIG. 11 contains a schematic block diagram illustrating a system forsupporting surgery, according to some embodiments.

FIG. 12 is a schematic block diagram of a hyperspectral imaging device,according to an alternative embodiment.

FIG. 13 is a flow chart of the method of depth resolution offluorescence.

FIG. 14 illustrates spectra of fluorescent light emitted at severaldepths in tissue and measured above a surface of tissue.

FIG. 15 is a flow diagram of a method of imaging.

FIG. 16 is a flowchart of correction of a fluorescence image forabsorption and scattering in a target, such as tissue.

DETAILED DESCRIPTION OF THE EMBODIMENTS

A hyperspectral imaging device is one that covers a broader range of theelectromagnetic spectrum than visible light alone, such as visible andinfrared light, and which is capable of resolving received light intomore than the traditional three colors.

A hyperspectral imaging device is capable of acquire full field of viewimages of the region of interest, such as the surgical field of view,similar to broad beam fluorescence imaging devices used for wide fieldimaging.

A hyperspectral imaging device is one that is capable of selectingwavelengths of interest in the visible and infrared regions of theelectromagnetic spectrum, and as such capable of acquiring multipleimages at user-defined wavelengths of interest

A hyperspectral imaging device is capable of pixel by pixel full spectrareconstruction using multiple images acquired at user-definedwavelengths of interest

Qualitative, single-bandpass, visible fluorescence imaging (vis-FI)during surgery has demonstrated improved visualization of disease,bringing with it a growing interest in fluorescence imaging for surgicalguidance, but not without limitations. The use of single bandpassdetection in current vis-FI systems does not take full advantage of thespectral specificity of fluorophores. For example, spectrally resolveddetection according to the present system enables un-mixing ofnon-specific autofluorescence in detected tissue to improve detectionaccuracy. Further, with the advent of fluorescence for intraoperativeuse and associated novel fluorophores for surgical guidance (forexample, protoporphyrin IX (PpIX), folate receptor targeting agents,blood pooling agents like fluorescein and indocyanine green),spectrally-resolved approaches enable simultaneous un-mixing of multiplefluorescent agents from the same fluorescent signal, thus enabling invivo imaging of multiple reporter dyes.

We have shown that quantitative assessments of tissue fluorescence,decoupled from the effects of tissue optical properties, enable us toperform accurate quantification of absolute levels of fluorescentbiomarkers in brain tissue. This quantitative approach significantlyimproves tumor detection, thus outperforming the diagnostic capabilitiesof standard vis-FI, and allowing us to detect tumor tissue that wouldhave otherwise been left undetected.

In an embodiment of an imaging system for use in surgery, quantitativehyperspectral fluorescence and reflectance imaging are provided for useduring surgical procedures. The system detects cancer tissue by i)measuring a fluorescent contrast agent in a tumor and ii) measuringreflectance-derived endogenous optical markers to improve detection ofdiseased tissue using a quantitative hyperspectral fluorescence andreflectance imaging device that integrates with current surgicalmicroscopes. In an alternative embodiment, quantitative hyperspectralfluorescence and reflectance imaging is provided for use independentlyor with another operating room instrument.

The system herein described provides near real-time quantitativehyperspectral fluorescence and reflectance imaging evaluation of tissueduring surgical procedures to determine levels of fluorescent contrastagent and additional optical biomarkers in tissue in a field of view.Such information allows the surgeon to make decisions as surgeryprogresses and in conjunction with the surgical workflow, enabling asafer, more accurate and more effective surgical procedure throughimproved extent of tumor removal. A metabolic precursor to a fluorescentcontrast agent—in an embodiment 5-aminolevulinic acid-inducedprotoporphyrin IX (PpIX) fluorescence, causes preferential accumulationof fluorophores in malignant and benign brain tumors. This system isapplicable to other fluorescent contrast agents that may also marktissue of different types, including tumor and stroma types, with highsensitivity and specificity. The present system uses a hyperspectralfluorescence and reflectance imaging device to determine fluorophorelevels, such as PpIX levels, and fluorophore locations, throughout thefield of view, giving the surgeon information on the levels offluorescent agent which has accumulated in tissue and the locations ofconcentrations in tissue.

The additional optical biomarkers resolvable by the system, include, butare not limited to, hemoglobin concentration and oxygen saturation.These biomarkers are of use separately or in conjunction withfluorophore assessments to help inform surgical decisions.

In addition to hyperspectral imaging through the visual field of thesystem, the system has a probe-based hyperspectral contact pointspectroscopy device, which interrogates 1 mm² of tissue at a time, andwhich also provides a significant improvement in tumor tissue detection.Some tumor tissue that does not display visible fluorescence usingcurrent surgical microscopes enabled for single-band fluorescenceimaging, can be detected with a hyperspectral point process. Thecombined fluorescence and reflectance probe and method, according toembodiments described herein, measures optical biomarkers, includingfluorophore concentrations, oxygen saturation, hemoglobin concentration,and scattering parameters, to improve tumor detection across a range ofglioma histologies.

Fluorescence signals are strongly affected by variations in fluorophoredepth, tissue absorption and transport scattering properties, i.e.,tissue optical properties. A previous in vivo contact point spectroscopytechnique quantifies fluorescence in tissue by compensating for theeffects of tissue optical property variation and quantifies additionaloptical biomarkers such as hemoglobin concentration, oxygen saturationand scattering parameters. This technique was described in, for example,Valdes, P. A., Quantitative fluorescence in intracranial tumor:implications for ALA-induced PpIX as an intraoperative biomarker.Journal of Neurosurgery, 2011. 115: p. 11-17, which is incorporatedherein in its entirety by reference.

Point spectroscopy quantitative fluorescence and reflectance results areextended to a broad-beam imaging platform in the present device.

The device sequentially acquires white light reflectance andfluorescence emissions over a broad range of wavelengths, in aparticular embodiment in the range of 400-720 nm for white lightreflectance and 550-720 nm for fluorescence when using, for example,PpIX. In alternative embodiments the wavelengths imaged by thehyperspectral imaging device may include additional wavelengths from 400to 1100 nm. A light absorption-scattering correction applied across thewhole imaged field of view uses reflectance data to correct the detectedfluorescence spectrum. The device uses a spectrally-constrainednormalization correction algorithm, in which the ratio of thereflectance near the region of the excitation wavelength and thereflectance near the region of the emission wavelength dividesintensities of raw spectrally resolved fluorescence data to provide acorrected fluorescence data.

Spectral decomposition is then used to extract individual fluorophorecontributions from the corrected fluorescence data, and quantify theconcentration of fluorophores in tissue, and the depth of concentrationsof fluorophores in tissue. In addition to the corrections required forfluorescence quantification, the reflectance data is used to retrieveadditional optical parameters, including but not limited to oxygenationfraction, oxy- and deoxy-hemoglobin, myoglobin, and bilirubinconcentrations, that may be of interest to a surgeon.

The devices herein described provide:

-   -   i) reflectance hyperspectral imaging to quantify        reflectance-derived optical parameters, including but not        limited to, oxygen saturation, oxy- and deoxy-hemoglobin,        myoglobin, and bilirubin concentrations, and spectral        distortions;    -   ii) reflectance hyperspectral imaging to determine correction        parameters for correcting fluorescence images for light        absorption and scattering effects; and    -   iii) quantitative hyperspectral fluorescence imaging. The system        provides near real-time, more accurate, quantitative, and        spectrally resolved fluorescence and reflectance assessments of        the whole field of view. The system provides combined use of        spectrally resolved reflectance and fluorescence for more        accurate tissue identification across the whole field of view.

We also show phantom, pre-clinical and clinical data where we acquirehigh resolution spectra for every pixel in the field of view innear-real time, resolve multiple dyes and other tissue-type informationintraoperatively, and provide non-subjective, absolute estimation offluorophore concentrations in tissue, improving detection limits by oneto two orders of magnitude. Overall, our technique can have broadimplications for fluorescence image guidance and improving extent ofresection, by enabling clinically feasible in vivo fluorophorequantification.

We have shown that quantitative assessments of tissue fluorescence,decoupled from the effects of tissue optical properties, enable us toperform accurate quantification of absolute levels of fluorescentbiomarkers in brain tissue. This quantitative approach significantlyimproves tumor detection, thus outperforming the diagnostic capabilitiesof standard vis-FI, and allowing us to detect tumor tissue that wouldhave otherwise been left undetected.

FIG. 1 is a schematic diagram of a through-microscope spectral resolvedquantitative fluorescence imaging system and a surgical microscopeenabled for fluorescence imaging, according to some embodiments. Thesurgical microscope includes two optical ports (1 and 2) associated withtwo respective oculars for two surgeons and a third optical port (3)that is coupled to a RGB CCD camera for video rate and still imageacquisition. In some embodiments, this CCD camera is a color orblack-and-white stereo camera usable for image acquisition or forsurface profile extraction using stereo-optic techniques. A fourth freeoptical port (4) allows for integration of the portable, quantitativefluorescence and reflectance imaging device (QFI) according to thepresent system.

In exemplary embodiments, the QFI includes: i) an optical component thatfits onto a surgical microscope, or which can be used with anotheroperating room instrument or used independently, ii) a high-speed,tunable, wavelength-selection device, for example, a liquid crystaltunable filter, iii) a camera, and iv) a digital image processor. In anembodiment, the tunable wavelength-selection device is a liquid crystaltunable filter such as described in Nourrit, V. et al., “High-resolutionhyperspectral imaging of the retina with modified fundus camera”. J FrOpthalmol 33(10), 686-692 (2010). In some alternative embodiments, thetunable wavelength-selection device is an acousto-optic tunable filtersuch as described in Fellers, T., et al., “Acousto-Optic Tunable Filters(AOTFs),” National High Magnetic Field Laboratory, 1800 East Paul DiracDr., The Florida State University, Tallahassee, Fla. 32310 (2004). Insome alternative embodiments, the hyperspectral imaging device includesa scanning line spectrophotometer which includes an entrance slit and arotating scanning mirror. The mirror scans the image over the slit to adispersive element, such as a prism or a diffraction grating. Atwo-dimensional image sensor receives a plurality of spectra along afirst dimension of the image sensor, with each spectrum being spreadalong the second orthogonal dimension of the two-dimensional imagesensor. Such scanning line spectrophotometers are described in Gao, etal., “Snapshot Image Mapping Spectrometer (IMS) with high samplingdensity for hyperspectral microscopy,” Opt. Express 18, 14330-14344(2010).

The microscope has an illumination device adapted to provideillumination in both a blue or stimulus light mode for fluorescenceimaging, and in a white light mode for white light reflectance imaging;wherein the stimulus light mode has a filter, or an emitter thatprovides specific wavelengths, such that light of fluorescencewavelength is excluded from the tissue. The QFI system connects to thefree optical port of the surgical microscope.

In the method illustrated in FIG. 2 , a hyperspectral fluorescence andreflectance imaging device is adapted to a surgical microscope.Hyperspectral fluorescence data are acquired, processed and displayed innear real-time during a surgical procedure, enabling a surgeon to viewimages representing quantitative levels of fluorescent agent andadditional optical markers across the whole field of view. Thequantitative information aids the surgeon in making surgical decisionsgiven the levels of optical markers present, for example, PpIX, oxygensaturation, etc.

Referring to FIG. 2 , the QFI provides two sets of 3-dimensional imagecubes (x,y,λ) in memory of the digital image processor, each image cubehaving a sequence of two dimensional images recorded by the QFI, eachimage of the sequence recorded at different wavelengths. The image cubesinclude a white light reflectance image cube under white light mode anda fluorescence image cube under a selected fluorescence excitation orstimulus light mode, such that for each (x,y) coordinate of a2-dimensional image of the image cubes, either a reflectance orfluorescence spectrum can be reconstructed.

FIG. 3 contains a functional block diagram of workflow of thequantitative fluorescence imaging of the present device, according tosome embodiments. Referring to FIG. 3 , both reflectance andfluorescence image cubes are acquired for each pixel coordinate (x,y).The reflectance spectra are used in a light attenuation correctionalgorithm to derive a correction map which accounts for the distortingeffects of tissue optical properties on the measured fluorescence ateach pixel coordinate (x,y). The correction map is applied to thefluorescence spectra at each pixel coordinate (x,y) to derivefluorescence spectra which have been corrected for the effects of tissueoptical properties. Spectral un-mixing techniques are applied todecouple the individual contributions of each fluorophore of interest,for example, autofluorescence, PpIX, to the corrected fluorescencespectra, and calculate a quantitative image of the absoluteconcentration of fluorescent biomarker in tissue, for example, c_(PpIX).

The spectrally-resolved quantitative fluorescence imaging system andtechnique of the present system described herein were designed forseamless integration with the flow of surgery and compatibility with thestate-of-the-art surgical microscopes. The imaging system is portableand affordable. In addition, the ability to spatially resolve down inthe submillimeter range, enables both pre-clinical and clinical studiesand ease of translation between each platform.

We explored the system's capabilities to quantify a commonly usedclinical fluorescent biomarker, PpIX, in tissue simulating phantoms. Wenoted highly accurate and sensitive PpIX biomarker quantification innear-real time compared to the raw fluorescence, F_(RAW) (0.89% vs.0.27%, R² linear regression coefficient), pointing to the ability ofmaking fluorescence imaging in surgery quantitative and highlysensitive. Furthermore, quantitative fluorescence imaging demonstratedimproved contrast compared to both vis-FI and F_(RAw), withapproximately 2 and 1 orders of magnitude larger dynamical range thanvis-FI and F_(RAW) respectively.

The imaging system provides significant improvements in intraoperativediagnostic fluorescence imaging. Further, the imaging system provides anideal pre-clinical-to-clinical testing platform for testing of novelimaging agents and pre-clinical models, with subsequent ease oftranslation from bench-to-bedside. The ease of application, seamlessintegration into the surgical workflow, portability, reasonable costs,and small footprints, allows for implementation across modern operatingrooms and visualization of multiple fluorescent biomarkers.

Results

Imaging System Design

The imaging system includes a set-up with a small footprint, whichallows for ease of implementation in the operating room. The maincomponents in a prototype embodiment include an optical adapter, aliquid crystal tunable filter (LCTF) (Cambridge Research Instruments),and a charge-coupled device (CCD) camera (Cooke Corp.) connected to adigital image processor and computer control system. The LCTF is awavelength selection device to enable acquisition of images at eachwavelength of interest and create the 3D spectrally resolved image cube.Other type of electronic camera, for example, EMCCD, ICCD, etc., or CMOSsensor cameras, which allow for collection of light and conversion todigital form, can also be used. The prototype embodiment liquid crystaltunable filter (LCTF) performs fast (50 ms) single band selection (7 nmFWHM) for filtration of incoming light in the range 400-720 nm. The LCTFdisplays a wavelength specific transmission response in the rangeλ=400-720 nm, with a max transmittance of 64% at 710 nm. The CCD camera(1396×1024 pixels; 2×2 binning; 62% quantum efficiency at 580 nm.

A fully integrated, functional system requires only standard maintenanceprovided to conventional surgical microscopes. The system is portable,which allows for ease of transportation between operating rooms, andinstallation into a commercial surgical microscope is seamless.

An optical adapter connects the main components (LCTF and camera) to anoptical port of a surgical microscope, and between LCTF and a c-mountCCD camera. An LCTF performs fast (50 ms) single band selection (7 nmfull width at half maximum) for filtration of incoming light in therange λ=400-720 nm. The LCTF displays a wavelength specific transmissionresponse at a selected wavelength in the visible light range (i.e.,λ=400-720 nm), with a maximum transmittance of 64% at 710 nm. The CCDcamera (1396×1024 pixels; 2×2 binning; 62% quantum efficiency at 580 nm,PCO.Pixelfly, Cooke Corporation) senses the incoming light and transmitsthe digital counts to a computer control system. In a particularexemplary illustration, we operated in the visible region of thespectrum, given that two of the three major clinically used fluorescentimaging dyes operate in this region (i.e., fluorescein, PpIX). Thesystem can be extended to the near infrared (650-1100 nm), given theadvantages of working in this range. With the advent of novel imagingagents in the near infrared (NIR), this set up could readily accommodateeither a LCTF operating in the NIR or a dual visible-NIR region LCTF.

Phantom Validation

We assessed the quantification capabilities of our QFI system. Wedeveloped a spectrally-constrained normalization algorithm that correctsthe detected, raw fluorescence from the distorting effects of tissueoptical properties. The varying attenuation due to tissue opticalproperties on both the excitation light and fluorescence emissionsimpact the detected fluorescence non-linearly. This varying attenuationwas corrected, enabling absolute quantification of fluorophores intissue independent of varying tissue specific attenuation. We usedpreviously validated tissue simulating liquid phantoms of varyingabsorption (μ_(a)) and reduced scattering (μ_(s)′) at the excitation andmain emission peak of PpIX (λ=405 nm and λ=635 nm, respectively) andwith varying PpIX concentrations in the range 20-5000 ng/ml—the commonlyfound range of PpIX concentrations (lower to higher end) found in normaland pathological tissues.

Current state-of-the-art clinical systems use single band pass detectionof the emitted fluorescence. We found from previous work and with ourphantom studies, that vis-FI was able to visually detect concentrationsno lower than ˜1000 ng/ml. Here we performed an area under the curveintegration of the raw optical signal in the range λ=610-720 nm, as ameans to measure the raw, uncorrected fluorescence signal typicallydetected using contemporary clinical systems, F_(RAW) We then used ourquantification technique on the spectrally-resolved wide filed data tocalculate the pixel-specific PpIX concentrations, c_(PpIX) and thusestimate absolute PpIX levels across the full field of view. Wedemonstrate in our tissue simulating phantom studies that ourquantitative imaging technique effectively deconvolves the non-lineardistorting effects of varying optical properties on the emittedfluorescence (R²=0.89, vs. R²=0.27, linear regression analysis).Further, quantitative fluorescence imaging (i.e., QFI) demonstratedimproved contrast compared to both vis-FI and F_(RAw), withapproximately 2 and 1 orders of magnitude larger dynamical range thanvis-FI and F_(RAW), respectively. These results demonstrate highlyaccurate and sensitive estimates of absolute PpIX levels in near-realtime imaging mode, and that our technique improves the limit ofdetection of fluorescence imaging by 1-2 orders of magnitude, pushingthe lower limit of PpIX found in tissues. This latter point is ofimportance, since it indicates that our imaging system can quantify low,yet diagnostically significant PpIX levels currently found in low-gradegliomas, the tumor histology where the impact of fluorescence imaging onpatient survival could be profound.

We tested the spectral-resolving power of our imaging system on tissuesimulating phantoms with homogeneously distributed and varying levels oftwo commonly used clinical fluorophores, PpIX and fluorescein. Oursystem was able to simultaneously detect and spectrally un-mix theindividual fluorescence emission spectra in imaging mode across the fullfield of view of both fluorophores at varying concentrations. Theseresults demonstrate the ability to do accurate simultaneousmultiple-tracer imaging.

System Spatial Resolution

We first tested the spatial resolution of the system using aconventional method to measure the contrast transfer function (CTF) froma USAF 1951 standard contrast resolution target. The imaging systemdemonstrated submillimeter resolutions both in the vertical (214 μm and125 μm) and horizontal directions (217 μm and 125 μm) (Rayleigh andSparrow criteria, respectively), which are approximately 5 to 10 timessmaller than the smallest length scale in which the surgeon operates.This indicates that the system demonstrates sufficient spatialresolution for both human as well as pre-clinical animal studies, whichoperate in the submillimeter to millimeter scales.

Spectrally-Resolved Quantitative Fluorescence Imaging Data.

Each quantitative fluorescence imaging sequence acquisition entailsacquiring a 3-dimensional image cube at a 5 nm wavelength resolution(range λ=450-720 nm) acquired under white light mode for reflectancefollowed by a 3-dimensional image cube at a 3 nm resolution (rangeλ=600-720 nm) acquired under blue or excitation light mode forfluorescence.

Each 3-dimensional image cube (x, y, λ) includes a series of2-dimensional images (x, y) (i.e., spatial dimensions) at a plurality Nof specific wavelengths (λ) (i.e., spectral dimension) in the range ofinterest acquired under white light mode for reflectance or under bluelight mode for fluorescence signals. That is, the image cube is a stackor series of 2-dimensional images taken at a plurality of wavelengthsettings of the tunable filter. Each individual pixel spatial coordinate(x, y) corresponds to a location on the field of view. We reconstructedhigh-resolution (i.e., 1 nm) full reflectance and fluorescence spectrafor each pixel coordinate using a cubic spline interpolation techniquefor a total of 361,920 spectra per image cube (696×520=361,920).

Tissue Attenuation Correction Algorithm.

The white light reflectance spectra are used to correct the detectedfluorescence spectra for the attenuation caused by tissue absorption andscattering. We used a spectrally-constrained normalization technique toestimate the intrinsic fluorescence in tissue independent of the effectsof tissue optical properties by calculating the intrinsic fluorescencevalue, Φ, with the relationship in Eq. 1,

$\begin{matrix}{{\Phi_{Corrected}^{Fluo}(\lambda)} = \frac{\Phi_{Raw}^{Fluo}(\lambda)}{\Phi_{x}^{Ref} \times \left( \Phi_{m}^{Ref} \right)^{\alpha}}} & \left( {{Eq}.\mspace{14mu} 1} \right)\end{matrix}$where Φ_(Raw) ^(Fluo)(λ) is the wavelength dependent, raw fluorescenceintensity; Φ_(x) ^(Ref) and Φ_(m) ^(Ref) are the reflectance signalsintegrated over the ranges λ=465-485 nm and λ=625-645 nm, respectively.The range for Φ_(x) ^(Ref) was determined as close to the excitationwavelength band (i.e., 405 nm) to approximate light attenuation atexcitation, and the range for Φ_(m) ^(Ref) was determined at the mainemission peak around λ=635 nm. Our correction algorithm makes theassumption that most of the light attenuation is due to absorption atthe excitation wavelength, given the high absorption due to hemoglobin,and that at the emission band, scattering dominates over absorption by1-2 orders of magnitude more and can be corrected by an empirical powerfunction of Φ_(m) ^(Ref).

We apply our tissue attenuation correction algorithm on each pair ofreflectance and fluorescence image cubes. We integrate for each pixelcoordinate (x_(i), y_(i)) its corresponding high resolution reflectancespectrum to calculate Φ_(x) ^(Ref) (i) and Φ_(m) ^(Ref) (i). We thenestimated the corrected fluorescence spectrum for each (x_(i), y_(i))pixel coordinate, Φ_(Connected) ^(Fluo) (i), by applying therelationship in Eq. 1 to each corresponding pixel coordinate (x_(i),y_(i)) high-resolution raw fluorescence spectrum, Φ_(Raw) ^(Fluo)(i). Anon-negative least-squares routine is used on the corrected fluorescencespectra to spectrally un-mix the individual contributions of mainfluorophores: PpIX, fluorescein, and tissue autofluorescence using thefollowing relationship,c ^(rel)=(B ^(T) B)⁻¹ B ^(T)Φ_(Corrected) ^(Fluo)  (Eq. 2)where B=[b₁ b₂ . . . b_(N)] is a matrix of basis spectra for Nfluorophore components (for example, PpIX, fluorescein,autofluorescence), and c^(rel) is a relative concentration fluorophorevector. A system-specific calibration factor derived from the leastsquares regression on phantoms of known PpIX concentrations is used toconvert the spectrally un-mixed relative fluorophore concentrationvalues into absolute biomarker(s) concentrations for each pixel,c ^(abs) =S _(cal) c ^(rel)  (Eq. 3)where S_(cal) is the system specific scalar calibration factor. We thenare able to display a quantitative image map of PpIX concentrations.Tissue Phantoms.

Liquid tissue simulating phantoms that simulate the range of opticalproperties found in normal brain and brain tumor using an absorber dye(McCormick) as the main absorber and Intralipid as the scattering mediumwere fabricated for a total of nine (9) phantoms of three (3) differentoptical absorption (μ_(a) ^(x)=20, 40, and 60 cm⁻¹) and scatteringvalues (μ′_(s) ^(m)=8.7, 11.4, and 14.5 cm⁻¹) for a total of nine (9)different phantoms. For each tissue phantom, nine (9) PpIXconcentrations covering the range found in brain tumors (0.019, 0.039,0.078, 0.156, 0.313, 0.625, 1.250, 2.500, and 5.000 μg/ml) were added,and a final set with non PpIX added for a total of 90 phantoms ofvarying absorption, scattering and PpIX.

We acquired a pair of reflectance and fluorescence image cubes for eachphantom under typical low-light operating room conditions used forfluorescence guided surgery. Each pair was processed for quantitativefluorescence imaging to estimate a PpIX concentration image map. Aregion of interest analysis of the corresponding (x_(i), y_(i)) pixelcoordinates of the raw fluorescence image cube and on the calculatedPpIX concentration image map for each phantom was used to (i) integratethe average area-under-the-curve intensity in the range 610-720 nm forthe raw fluorescence spectra (i.e., ∫_(610 nm) ^(720 nm)Φ_(Raw)^(Fluo)(λ) dλ), F_(RAW) and (ii) the average PpIX concentration,c_(PpIX), for the quantitative image map. The former corresponds to theraw fluorescence signal of PpIX, and is equivalent to using a bandpassfilter to filter out only the raw fluorescence in that range ofinterest.

F_(RAW) and QFI derived c_(PpIX) for each phantom were plotted versusthe true known PpIX concentrations of each phantom. A linear regressionanalysis was then used to ascertain (i) the linearity of fluorescencewith fluorophore concentration, which provides a measure of thenon-linear effects of optical properties and the correction capabilitiesof our algorithm; and (ii) the limit of detection of both approaches,which provides a measure of the sensitivity of each approach to detectlow levels of fluorophore.

A normalized contrast for the raw fluorescence and our quantitativeapproach was calculated by dividing F_(RAW) or c_(PpIX) at everyconcentration from i=5.000−0.039 μg/ml by the F_(RAW) or C_(PpIX) at theminimum concentration, 0.019 μg/ml, respectively,

$\begin{matrix}{\left\langle {{{nContrast}_{RAW}(i)} = \frac{F_{RAW}^{i}}{F_{RAW}^{\min}}} \right\rangle\mspace{14mu}{or}\mspace{14mu}\left\langle {{{nContrast}_{QFI}(i)} = \frac{{QFI}^{i}}{{QFI}^{\min}}} \right\rangle} & \left( {{Eq}.\mspace{14mu} 4} \right)\end{matrix}$System Spatial Resolution

A USAF 1951 (Edmund Optics) contrast resolution target was used todetermine the spatial resolution of the imaging system. We imaged theUSAF 1951 target under white light mode, which is made up of n=[0, 1, 2,3] groups, each with i=[1, 2, 3, 4, 5, 6] horizontal and verticalelements consisting of black and white bar patterns. We extracted thecross-sectional intensity profiles for each element to calculate theelement-specific contrast transfer function (CTF) using the followingrelationship,

$\begin{matrix}{{{CTF}(i)} = {\frac{{I_{\max}(i)} - {I_{\min}(i)}}{{I_{\max}(i)} + {I_{\min}(i)}}*100}} & \left( {{Eq}.\mspace{14mu} 5} \right)\end{matrix}$where I_(max) and I_(min) are the maximum and minimum intensities of thebar pattern's cross-sectional profile. We matched the calculated elementspecific resolution using the following relationship

$\begin{matrix}{{{Resolution}(i)} = {\frac{1}{2^{n + \frac{i - 1}{6}}}*1000}} & \left( {{Eq}.\mspace{14mu} 6} \right)\end{matrix}$where n is the group number, i is the element number, and resolution isgiven μm. We calculated the Rayleigh and Sparrow horizontal and verticalresolutions by determining the point on the graph were the CTF=26.4% orCTF=0%, respectively.Data Processing.

Data processing and analysis was done using custom MATLAB® software(Version R2010a, The Mathworks, Inc., Natick, Mass., USA)

Statistical Analysis.

Statistical analyses were performed with MATLAB. Linear regressionanalysis was used to determine the coefficient of determination, R².Non-parametric, Wilcoxon rank-sum (Mann-Witney) tests were used tocompare differences. Two-sided P<0.05 are described as statisticallysignificant.

Experiments provided absolute quantification of tissue concentrations ofthe metabolic biomarker protoporphyrin IX (PpIX) following systemicadministration of the prodrug 5-aminolevulinic acid (ALA) usingfiberoptic-based point fluorescence and diffuse reflectance spectroscopyin vivo. This technique significantly improves the clinical utility fordetection of residual tumor during neurosurgical resection of variouspathologies, including high-grade and low-grade gliomas.

In a point spectroscopy device referred to as the “Standard Probe”, thefull fluorescence spectrum is measured single points at the tissuesurface, together with the full diffuse reflectance spectrum. Aspectrally-constrained diffusion theory model is then used to determinethe absorption and transport scattering spectra of the tissue (μ_(a) andμ′_(s), respectively) and these values are then used to correct themeasured fluorescence spectrum yielding the true fluorescence signal.Data has been taken with the Standard Probe, this data has been used toconfirm practicability of the hyperspectral imaging device describedherein, and to test practicability of the spectrally—constrainednormalization approach described herein. The local concentration of PpIX(C_(PpIX)) is then determined by spectral unmixing to eliminatecontributions from PpIX photoproducts and tissue autofluorescence. A nonmodel-based spectrally-constrained normalization approach is describedherein. The potential advantages are that it simplifies theinstrumentation, since only two reflectance detection channels arerequired. It is expected that the hyperspectral imaging device hereindescribed will be able to provide similar information when used in vivo.

FIG. 4 is a schematic functional block diagram of the intraoperativestandard probe system according to some embodiments. Referring to FIG. 4, measurements are made with three adjacent optical fibers makingcontact with the tissue at the distal end of a 1.1 mm diameterintraoperative probe. Each fiber has a core diameter of 200 μm and thecenter-to-center separation between adjacent fibers is 260 μm. One fiberis coupled to a broadband light-emitting diode (LED) delivering lightfrom λ=450 to 720 nm. Another fiber is coupled to a violet LED withcenter wavelength at λ=405 nm. The third fiber is coupled to a miniaturefiber optic spectrometer (USB2000+, Ocean Optics Inc., FL) with lightdetection capabilities between λ=200 nm and 1100 nm at a spectralresolution <˜1 nm. The PpIX absorption spectrum exhibits a strongabsorption band (Q-band) around λ=405 nm, so that fluorescenceexcitation is efficiently achieved using the violet light source fiber.The effective tissue sampling depth is <˜1 mm. White light reflectanceand fluorescence emission spectra are sequentially measured usingbroadband and light excitation at λ=405 nm, respectively. FIG. 5 showstypical spectra acquired during a surgical resection of a high-gradeglioma. That is, FIG. 5 illustrates the PpIX fluorescence spectrum(left) and the diffuse reflectance spectrum (right) measured in ahigh-grade glioma following tissue excitation with blue and white light,respectively. The vertical bands in the diffuse reflectance spectrum(right) indicate the detection windows used in the normalizationmeasurements.

Spectrally-constrained normalized values, Φ^(Ratio), were computed usingthe formula

$\begin{matrix}{{\Phi^{Ratio}(\lambda)} = \frac{\Phi^{Fluo}(\lambda)}{\Phi_{x}^{Ref} \times \left( \Phi_{m}^{Ref} \right)^{\alpha}}} & \left( {{Eq}.\mspace{14mu} 7} \right)\end{matrix}$where Φ^(Fluo) is the raw measured fluorescence spectrum, and Φ^(Ref)_(x) and Φ^(Ref) _(m) are the spectrally-integrated reflectance signalsin the range λ=465-485 nm and λ=625-645 nm, respectively. The wavelengthrange for Φ^(Ref) _(x) was chosen to be as close as possible to thefluorescence excitation band in order to correct for tissue attenuationof the excitation light. The second band samples the main PpIXfluorescence emission peak around λ=635 nm. The underlying assumptionsof Eq. (7) are: (1) that most of the spectral distortion effect is dueto tissue absorption at the excitation wavelength. and (2) that theimpact of light diffusion in the emission band, where scatteringdominates over absorption, can be corrected by further dividing the rawfluorescence using an empirical power function of Φ^(Ref) _(m).

Three separate sets of measurements with PpIX were conducted to evaluatethe accuracy of the spectrally-constrained normalization algorithm: (1)in tissue-simulating phantom experiments with a range of opticalproperties consistent with human brain tissue, (2) using ex vivo animalbrain tumor tissue, and (3) using in vivo clinical data previouslyacquired with the Standard Probe during resection procedures in braintumor patients. For each measurement, a background spectrum was acquiredand subtracted from the raw fluorescence in order to account for thedark noise of the instrument, and the resulting spectrum was correctedusing Eq. (7) for values of the empirical parameter α ranging from −2 to2. The corrected spectra were then unmixed to separate the contributionsof the main fluorophores, namely PpIX, its photoproducts and tissueautofluorescence. The resulting pure PpIX spectra were then normalizedto a calibration factor, which in turn was derived from the slope of thelinear regression fits in phantoms with known dye concentrations, inorder to extract an absolute PpIX concentration, C_(PpIX).

The phantoms comprised Intralipid (Fresenius Kabi, Uppsala, Sweden)which provided the tissue-like background scattering, yellow foodcoloring (McCormick, London, Ontario) as the absorber and PpIX. Ninephantoms of differing optical properties were prepared, and a total ofsix concentrations of PpIX (0.15625, 0.3125, 0.625, 1.25, 2.5, and 5μg/ml) for each phantom were mixed, giving a total of 54 separatephantoms with different absorption coefficient (μ_(a)), reducedscattering coefficient (μ_(s)′) and fluorophore concentration. In orderto mimic human brain tissue, the absorption and reduced scattering atthe excitation wavelength ranged from μ_(a)=20 to 60 cm⁻¹ and μ_(s)′=15to 25 cm⁻¹, respectively, and at the emission wavelength ranged fromμ_(a)=0.02 to 0.06 cm⁻¹ and μ_(s)=8.7 to 14.5 cm⁻¹. Fluorescence andreflectance measurements were made for each phantom using the StandardProbe. FIG. 6A-6F illustrate raw and corrected emission spectra measuredfor tissue phantoms, according to some embodiments. Three different PpIXconcentrations and a range of absorption and scattering values wereused: (FIG. 6 a , FIG. 6 b , FIG. 6 c ) varying absorption with constantscattering; (FIG. 6 d , FIG. 6 e , FIG. 60 varying scattering withconstant absorption. The value of α=−0.7 is used in FIG. 6 c and FIG. 6f Illustrated is the marked effects of varying absorption (FIG. 6(a) andscattering (FIG. 6(b)), on the measured fluorescence spectra, primarilyin the intensity, with some distortion also of the spectral shape. Intissue, the latter is greatly due to the strong spectral dependence ofhemoglobin absorption. Dividing the measured spectrum by Φ^(Ref) _(x)(FIG. 7 b ) partially corrects for changes in absorption and, to alesser extent, for changes in scattering (FIG. 7 e ). Normalization to afunction of Φ^(Ref) _(x) and Φ^(Ref) _(m) as in Eq. (7) can furthercorrect for changes in both absorption (FIG. 7 c ) as well as scattering(FIG. 70 , and yield quantitative fluorescence spectra that are largelyindependent of variations in the optical properties. As shown below, theoptimal tissue-specific empirical parameter α is determinedexperimentally using the tissue-simulating phantoms (or, for example,Monte Carlo modeling) in order to achieve the highest accuracy inC_(PpIX). As in previous studies, the spectral unmixing algorithm wasapplied to the corrected fluorescence spectrum, using the known PpIXemission spectrum as input. A calibration factor, derived from thelinear regression fit of Φ^(Ratio)(λ) against the true concentration wasthen applied to the pure PpIX spectrum to determine C_(PpIX). FIG. 7 andFIG. 8 illustrates quantification of PpIX concentration in phantomsbased on FIG. 7 raw fluorescence intensity (λ=635 nm), and FIG. 8estimated C_(PpIX) levels derived following correction of the rawfluorescence, according to some embodiments. FIG. 7 and FIG. 8 shows, asa function of the true PpIX concentration, both the raw fluorescenceintensity and the calculated C_(PpIX) following correction only for thestrong absorption effects at the excitation wavelength. There is asignificant improvement in the goodness-of-fit, from R²=0.639 (RMSE=72%)for the detected fluorescence (Φ^(Fluo)) to R²=0.944 (RMSE 24%) using acorrection by the reflectance close to the excitation only(Φ^(Fluo)/Φ^(Ref) _(x)). Correction by the product of Φ^(Ref)_(x)Φ^(Ref) _(m) ^(α) provided the greatest improvement, with R²=0.984(RMSE 13%), using α=−0.7 (FIG. 9(a)). This algorithm is applied in thesubsequent ex vivo tissue and clinical studies. FIG. 9 illustrates (a)empirical determination of the optimal α value for PpIX quantificationand (b) relationship between the estimated an actual PpIX concentrationin phantoms following correction of the raw fluorescence, as describedherein.

The spectrally constrained normalization method was then evaluated interms of its efficacy to quantify C_(PpIX) in biological tissue. 1×10⁶cells from the human glioma VX2 line were orthotopically implanted to adepth of 2 mm in the brains of five mice. Following three weeks ofincubation, 100 mg/kg body weight of ALA (Sigma-Aldrich Inc., MO) wasinjected via the tail vein, and the animals were subsequently sacrificedby cervical dislocation under anesthesia at 0.5, 1, 2, 3 and 4 h later,respectively. The brains were then removed intact and measurements madewith the Standard Probe at three different locations for each sample.The probe measurements (fluorescence and reflectance) were used tocompute the ratio in Eq. (7) for α=−0.7, from which the C_(PpIX) valuewas calculated, as above. For comparison, quantitative fluorimetricmeasurements of C_(PpIX) were made on ex vivo brain tissue samples usingan established technique. Statistical analysis (paired Student's t-test)showed no significant difference between the fluorimetric measurementsand ratiometric estimates (p=0.33). These results represent an initialdemonstration that absolute quantification of PpIX in tissue using theratiometric approach according to some embodiments is feasible andaccurate.

The Standard Probe has been used in ALA-induced PpIX fluorescence guidedresection of human brain tumors. Here, we revisit the same clinical dataand retrospectively apply the ratiometric algorithm in patients with adiagnosis either of glioma (N=5), meningioma (N=6) or metastatic braintumors (N=3). Receiver operating characteristic (ROC) curves weregenerated to assess the performance of the normalization approach fordetecting abnormal tumor tissue. FIG. 10 contains ROC curves quantifyingthe performance of three different fluorescence measurement schemes foroptical datasets acquired in vivo during 14 brain tumor resectionprocedures. FIG. 10 shows the ROC plots of the quantified C_(PpIX) usingtwo ratiometric approaches compared with the raw measured fluorescence(Φ^(Fluo)) intensity peak at λ=635 nm for each tumor type. There was ahighly statistically significant difference in the performance of thenormalization approach, with the area under the curve (AUC) increasingto 0.92 (STD=0.03) from the value of 0.60 (STD=0.05) for the rawfluorescence signal.

The use of ALA-induced PpIX fluorescence guidance during tumor resectionis gaining importance in the neurosurgical community, and there isincreasing evidence that quantitative techniques that augment the visualfluorescence assessment of the surgeon yields clinical benefit. Ourprevious work demonstrated this using a full-spectrum device andalgorithm for quantitative point measurements. According to someembodiments, an optimized non model-based spectrally-constrainednormalization approach is used for PpIX quantification, applying asemi-empirical algorithm to correct for the distorting effects ofoptical attenuation in tissue. The advantage is that this is technicallysimpler in point-mode but is also translatable into wide-fieldquantitative fluorescence imaging mode more easily than thefull-spectrum approach. A possible limitation compared to the StandardProbe method, which is, at least in principle, universal and does notrequire tissue-dependent calibration, is that it requires experimentaldetermination of the empirical optimization parameter, α, as well as acalibration factor derived from the linear regression fit using phantomsof known concentrations. Further detailed studies are required toestablish any other limitations to the accuracy for absolute fluorophorequantification in vivo and this work is in progress, as is initialtesting in wide-field quantitative imaging mode.

These approaches to determining C_(PpIX) in vivo (or the concentrationof other exogenous molecules) may also be of value in applications suchas photodynamic therapy in order to monitor and optimize the dose ofphotosensitizer and the drug-light time interval for treatment.

FIG. 11 is a schematic block diagram illustrating a system 100 forsupporting surgery, according to some embodiments. The system of FIG. 11includes a microscope body 102, which has multiple beam splitters 104that permit light to be diverted to several optical portssimultaneously. Attached to a first optical port of body 102 is a tube106 leading to a surgeon's binocular optical eyepieces 108.

Attached to a second optical port of body 102 are a first highdefinition electronic camera 120 and a second high definition electroniccamera 122. Cameras 120, 122 are coupled to provide images to imagecapture interface 124 of a digital image processing system 126. Attachedto a third optical port of body 102 is a hyperspectral imaging device128 that in an embodiment has a tunable filter 130 adapted to receivelight from body 102 and a high resolution broad-bandwidth electroniccamera 132. The camera 132 of the hyperspectral imaging device 128 isalso coupled to provide images to image capture interface 124 of thedigital processing system 126. In an embodiment, tunable filter 130 is aliquid crystal tunable filter. In an alternative embodiment, tunablefilter 130 is an acousto-optic tunable filter.

FIG. 12 is a schematic block diagram of a hyperspectral imaging device,according to an alternative embodiment. In an alternative embodiment,hyperspectral imaging device 128 of FIG. 11 is replaced with thehyperspectral imaging device of FIG. 12 . In the hyperspectral imagingdevice of FIG. 12 , light received through optical interface port 202 isscanned by a rotating mirror 204 across a spectrometer slit 206. Lightpassing through spectrometer slit 206 is dispersed according towavelength by a dispersive device 208, such as a prism or a diffractiongrating, and thence onto a planar image sensor 210 such that each columnof planar image sensor 210 provides a spectrum of light received at acorresponding point in slit 206, and light received at slit 206corresponds to light scanned from a row of a virtual image receivedthrough microscope body 102.

Referring again to FIG. 11 , a tracker interface 140 of the imageprocessing system 126 is coupled to use sensors 142 rigidly attached toa reference location within an operating room to track locations ofmicroscope location sensors 144 and patient location sensors 146.Microscope location sensors 144 are rigidly attached to the microscopebody 102, and patient location sensors 146 are attached to a frame 148that may be attached to a patient while the patient is undergoing asurgical procedure. In a particular embodiment, frame 148 is adapted tobe attached to a patient's skull 150 by screws (not shown) for theduration of a neurosurgical procedure during which the patient's brain152 is exposed, and during which patient's brain 152 may be operated onwith surgical instruments 154 to remove one or more lesions 156.

Microscope body 102 also has zoom optics 160, adapted for operation by azoom motor/sensor 162, and a focus adjustment (not shown) adapted foroperation by a focus motor (not shown). The microscope also has multipleilluminators 166, 168. In an embodiment, illuminators 166, 168 includewhite-light illuminators 166, and fluorescent stimulus illuminators 168,operating under control of an illumination interface 170 of the imageprocessing system 126. The microscope body also has a heads-up display(HUD) projector 172 capable of providing graphical images through acombiner 174 of body 102 such that the graphical images are presentedfor viewing by a surgeon through surgeon's eyepieces 108. The surgeon'sfield of view through the operating microscope and its associated HUD isco-registered with that of the imaging system, allowing display oftissue classifications and hyperspectral imaging results superimposed onvisible brain tissue, one-to-one comparisons, and intraoperativesurgical decision making. At standard working distances betweenmicroscope and surgical cavity, surgical tools 154 fit between objective160 and tissue 152.

Image processing system 126 also has a memory 178 into which imagecapture interface 124 saves images received from cameras 120, 122, 132;and at least one processor 180. Processor 180 is adapted for executingprocessing routines such as surface profile extraction routines 182,brain deformation modeling routines 184, fluorescence depth modelingroutines 186, and hyperspectral image processing routines 188 stored inmemory 178 and operable on images stored in memory 178. Processor 180 isalso adapted for preparing images for display through display interface190 onto monitor 192, and for communicating through network interface194 to server 196; server 196 has a database containing informationderived from preoperative MRI and CAT scans 198.

Among the image processing routines in memory of image processingsubsystem 126 also has a classifier for classifying tissue typesaccording to a combination of fluorescence and backscatter information.In a particular embodiment, the classifier is a K-nearest-neighborclassifier.

Image processing system 126 is also interfaced to a handheld backscatterfluorescence probe 143 that acts as a single-channel, single-pointsubset of the fluorescence imaging microscope. Probe 143 has white 145and fluorescence stimulus 147 illuminators, and a spectrophometricdetector 149, the illuminators 143, 145, and detector 149 coupledthrough an optical fiber and lens subsystem 151 to apply light to, andread light from, a single point on tissue surface. The probe couplesinto the image processing subsystem 126 through a probe interface 153.

Operation of the system 100 has several modes, and each mode may requireexecution of several phases of processing on processor 180, executingone or more of several routines, as mentioned above. For example,processor 180 can execute the hyperspectral image processing routine toperform the hyperspectral fluorescence and reflectance imaging of thesystem, as described above in detail. The hyperspectral fluorescence andreflectance imaging can also be performed in connection withstereo-optical extraction, using images captured by stereo cameras 120,122, to perform tissue surface extraction and modeling, as described inU.S. Provisional Patent Application No. 61/583,092, filed in the UnitedStates Patent and Trademark Office on Jan. 4, 2012, and incorporatedherein in its entirety by reference. The tracking using tracking sensor142, patient location sensors 146 and microscope location sensors 144described above can be used in performing the brain deformation modelingroutine. The hyperspectral fluorescence and reflectance imaging can alsobe performed in connection with fluorescence depth modeling, asdescribed in U.S. patent application Ser. No. 13/145,505, filed in theUnited States Patent and Trademark Office on Jul. 2, 2011, andincorporated herein in its entirety by reference, and described below indetail, where fluorescence and reflectance spectral information isderived from hyperspectral imaging device 128. It should be noted thatin the referenced U.S. patent application Ser. No. 13/145,505 anddescribed below in detail, the depth modeling is described as using aratio of images at two wavelengths. Depth modeling using thehyperspectral fluorescence imaging of the present system can be improvedwith a modified process in which several ratios are used simultaneouslyusing a fitting routine.

A phase-3 human trial of surface fluorescent microscopy during surgicalremoval of malignant tumors marked with 5-ALA showed increased survivalfollowing surgery when all fluorescent tumor visible at excision cavitysurfaces are removed; see Stummer, et al., Fluorescence-guided surgerywith 5-aminolevulinic acid for resection of malignant glioma: arandomized controlled multicentre phase III trial, Lancet Oncol., 2006.7(5): p. 392-401, published online at oncology.thelancet.com.

5-ALA may also be usable to locate tumors in other organs. Inparticular, 5-ALA has been shown to be preferentially metabolized intoprotoporphyrin IX in tumors of the prostate as compared toprotoporphyrin IX production in surrounding normal prostate tissue. Thismay offer the opportunity to remove the prostate tumors with reducedrisk of damaging nerves responsible for penile sensation and erection;thereby preserving functions thought important by many patients.

In yet another embodiment, dye administered to a patient to locate atumor contains a fluorescent molecule or prodrug metabolized intofluorescent molecules in-vivo that is attached to a monoclonal antibodyhaving specificity for tumor tissue.

In an embodiment tailored for use with tumors metastatic from a distantorgan (not shown) such as lung, dye administered to a patient comprisesa fluorescent molecule or prodrug attached to an antibody, such as amonoclonal antibody, having specificity towards a tissue type of thedistant organ. Because many metastatic tumors retain some cell-surfacemarkers similar to those of the parent tissue, dye preferentiallydelivers the fluorescent molecule or prodrug to the tumor tissue, whiledelivering a substantially lower concentration of the fluorescentmolecule or prodrug to the organ.

A surgeon visualizes the tumor through the hyperspectral imaging devicedescribed herein.

Typically, tumors and organs are light propagation media whereabsorption and scattering is particularly important at shortwavelengths, making them seem opaque to visible light. Consequently, thewhite light images give good resolution but do not give good visibilityof features beneath a surface of the tissue of tumor and organ. Inparticular, light at the blue end of the visible spectrum is oftenabsorbed and scattered within less than a millimeter of the surface,rendering it difficult to see deeper structures. Further, because fineinvasions of tumor may have color and structure superficially resemblingthat of surrounding tissues, these fine invasions of tumor at thesurface may not be readily visible to the surgeon. However, theinteraction of far-red and near-infrared light with biological tissue issuch that propagation is inherently different when compared to that ofvisible light. For far-red and near-infrared light, scattering is stillimportant but absorption by the main contributing chromophores such ashemoglobin and water are dramatically reduced (see FIG. 23 ). Thisprovides an opportunity for imaging structures embedded deeper into thetissue using far-red and near-infrared light.

In an alternative embodiment, the monochromatic illuminator provides redor near infrared light between 600 and 1000 nanometers wavelengths, thislight is capable of deeper penetration into organ and tumor than blue orgreen light and is within the range of wavelengths that can be detectedby available electronic cameras. The wavelength used is chosen to beappropriate for the fluorophore used to label the organ or tumor.

The hyperspectral imaging device herein described may, in an embodiment,be operated to provide basic fluorescent imaging. This is done byilluminating the tissue with a light at a stimulus wavelength, in aparticular embodiment by illuminating with a blue light-emitting diode(LED) having a filter to exclude illumination at a fluorescence emissionwavelength. The tunable liquid-crystal filter is then set to pass onlylight at the fluorescence emission wavelength. This provides the surgeonwith non depth-resolved information pertaining to fluorophoredistribution beneath the surface; this information may be somewhat hazydue to scattering in the tissue.

In an embodiment, multiple monochromatic illuminators are provided, andthe tunable liquid crystal filter adjusted appropriately, permittingrapid selection of surface and deep imaging wavelengths, and wavelengthssuitable for different fluorophores.

The surgeon views the fluorescence image information on monitor 192before, after, or at intervals while using surgical instruments such assurgical tools 154 to remove or destroy part or all of tumor 156.

When the surgeon wants to determine if any tumor tissue remains in theorgan 152, the surgeon triggers the system to obtain new fluorescencetomographic image information, views that information of monitor 192,and may remove additional portions of tumor 156 and organ 152.

In an embodiment, blue or ultraviolet monochromatic stimulus-wavelengthemitters have peak emission between three hundred fifty and four hundredfive nanometers to induce visible fluorescence from protoporphyrin IX intumor. Alternative embodiments may replace blue or ultraviolet floodemitters with, or provide an additional set of monochromatic floodemitters, for providing stimulus specifically appropriate to afluorophore other than protoporphyrin IX present in tumor. For example,near-infrared monochromatic emitters may be used with other dyes such asthose based on indocyanine green. In an embodiment, monochromaticemitters are LEDs; LEDs are available in a wide variety of spectralcharacteristics, can provide intense light, and are small and easy touse. In an alternative embodiment, monochromatic emitters are a tungstenhalogen or high intensity discharge light source equipped with amonochromatic filter for transmitting a wavelength appropriate forinducing fluorescence in fluorophores of, or metabolically producedfrom, the dye in use. Such an alternative light source may also includeoptical fibers and lenses for directing light from the monochromaticfilter to the tissue and tumor.

In some applications of the microscope other than open brain surgery,there may also be a matching liquid applied to the subject to give thearea being interrogated a geometry that is simpler to model.

In an embodiment, a left and right stereo image pair is captured 402.This image pair is used by the processor 180 to extract a surfaceprofile of the tissue. Image cubes are then captured with thehyperspectral imaging device and placed in memory as previouslydescribed. A computer model of diffuse scattering in the tissue is thenconstructed.

The tomography reconstruction algorithm is based on the assumption thatthe received light from the tissue is diffused, resulting fromscattering and absorption at each voxel in a model of the tissue. Eachvoxel is assigned a scattering parameter, absorption parameters forstimulus and emissions wavelengths, and fluorescence parameters, such asconcentration and fluorophore lifetime. Further, it is known that anemissions spectrum of light emitted from fluorophores in tissue isdistorted by passage through tissue by absorption by chromophorespresent in that tissue, details of the distortions induced beingcorrelated to depth of the fluorophores in tissue as illustrated in FIG.14 . Since this distortion due to depth effectively alters a “color” ofreceived light at the QFI, this color for each pixel of the image cubemay be determined from ratios of received light intensity in each ofseveral of the 2-D images of the image cube captured at differentwavelengths near peak fluorescent emissions. For example, shouldspectral distortions resemble those of FIG. 14 , ratios, and thus“colors”, may be computed from image data acquired at multiplewavelengths in the 680 to 710 nanometer range.

Voxels external to the modeled surface contour of the tissue haveabsorption, scattering, and fluorescence parameters assumed to beconstant, known, and zero in most cases to set boundary conditions onthe model of the tissue during the process of solving the differentialequations associated with light transport. Voxels within the organ,tumor, or tissue are initially assumed to scatter light and haveabsorbance as per an average absorption and scattering of biologicaltissues of the tissue type being observed. In some applications theaverage values of these properties are chosen based on literaturevalues.

The model is constrained to produce image data of “colors,” or ratiosbetween light received at each of several wavelengths, of intensitymatching data at selected wavelengths in the fluorescent image cube.

In this embodiment, optical diffuse tomography images of absorbance maybe displayed to the surgeon, showing concentrations of chromophores inthe tissue. These images may be of use during surgery in locating hiddenstructures such as blood vessels, aneurysms, and other concentrations ofheme.

Once the absorption and scattering parameters are refined, or inembodiments displaying fluorescent parameters only a default absorptionand scattering parameters may be used; for voxels within the tissue anillumination at each voxel is computed for each position of the incidentbeam using a computer-based light propagation model of the tissue andorgan.

Next, the fluorescence concentration parameters of each voxel arecomputed by using a least-squares fit of the fluorescence parameters tothe captured fluorescence image data. The fluorescence parameters thuscomputed for each voxel form a three-dimensional model of fluorophoredistribution in the tissue.

A sequence of tomographic images is constructed and displayed 414 byconsidering intersections of a plane with fluorescence distribution ofthe three-dimensional model of fluorescence distribution. In anembodiment, these tomographic images are displayed on monitor 192 and/orthe HUD to the surgeon as a sequence of increasing depth, the sequencebeing preceded by a white light photographic image of the organ orcavity for reference.

In an alternative embodiment, the tomographic images of increasing depthare encoded by coloring each image with a false color ranging from bluefor images at the organ surface to red for images more than a centimeterdeep into the surface. These images are then summed. The sum image,having fluorescence intensity encoded as intensity and fluorescencedepth encoded as color, is then displayed on display device. White lightimages and images of reflected and scattered laser light may also bedisplayed on display device to provide the surgeon with a visualreference.

Once the laser scanning is complete, and while image processingproceeds, white light illuminator is turned back on. Once the surgeonstudies the tomographic images on display device, the surgeon may removeadditional tumor or other organ tissue and repeat the processillustrated in FIG. 13 .

In an embodiment, computer simulation of the microscope shows thatfluorescence of protoporphyrin IX generated by 5-ALA tagging of tumortissue can be resolved to about two and a half millimeter resolution toa depth of twelve millimeters in organ. Further, presence of strongconcentrations of protoporphyrin IX can be detected to a greater depththan that at which this resolution can be obtained.

Studies of a mouse model of glioma show that removal ofprotoporphyrin-IX-tagged tumor tissue detected at the surface of anoperative site with a surface fluorescence microscope enhance survival;this has also been shown to be true by Stummer for human glioma. It isexpected that microscope will result in both improvements in survivalbeyond those obtained with surface fluorescent instruments and areduction in post-surgical neurological impairment.

It is also expected that the microscope will be of use in treating othercranial tumors such as meningioma, pituitary tumors, acoustic neuroma,and some metastatic tumors. It is expected that the microscope will alsobe of use in surgical treatment of tumors in other sites, includingskin, breast, liver, bowel, thyroid, eyes, pancreas, kidney, bladder,prostate and muscle, as well as some lesions of other types includingvascular abnormalities.

The microscope is also of use with indocyanine green for visualizationof aneurismal vasculature obscured by overlying vessels, aneurysms, orother tissue.

Light as the term used herein includes electromagnetic of the visible,near ultraviolet and near infrared portions of the spectrum that may befocused with lenses and imaged with readily available semiconductorimaging devices.

While the invention has been described with reference to fluorescentsubstances such as indocyanine green, protoporphyrin IX, andFluorescein, the apparatus and methods herein described are applicableto other biocompatible fluorescent prodrugs, dyes and molecules such asare known or may be developed in the future, and which may tend toconcentrate in tumor tissue to a different extent than in normal tissue.

An alternative embodiment of the method uses the fluorescent tomographicmicroscope heretofore described to visualize a fluorescent chromophoresuch as may occur naturally in some types of tumor tissues. In thisembodiment, there is no need to administering a prodrug or a drug. Thefluorescent tomographic microscope may also be used to visualize tumorsor tissues having concentrations of such a naturally occurringfluorescent chromophore that differ from concentrations in nearby normalorgan stroma. Nicotinamide Adenine Dinucleotide Hydride (NADH) is anexample of a naturally-occurring fluorescent chromophore that iscritical to function of mitochondria. As such NADH occurs in alleukaryotic tissues, including human organ stroma and tumor tissues, butmay be present at different concentrations in tumor and nearby normalstroma. Metabolically inactive tissues, such as fatty tissue of breast,may have very low concentrations of NADH, while adjacent malignant tumortissue may be metabolically very active and therefore possess largequantities of NADH; this difference in concentration permits using thefluorescent tomographic microscope to visualize the malignant tumortissue.

In this embodiment, scattered, reflected, and fluorescent light 1336from organ 1306 and tumor 1344 passes through a lens of lenses 1308, acollimating optic 1338, a filter system 1340, and into a high resolutionCCD camera 1342. Filter system 1340 has transparent modes for use withmultiple wavelengths for imaging heme, as well as excitation-wavelengthblocking and fluorescent-light passing modes for use when imagingfluorescent chromophores. In an embodiment, CCD camera 1342 is activelycooled to provide reduced noise and enhanced sensitivity to infraredlight.

In operation, in a heme imaging mode, images are captured by CCD camera1342 at each of the three wavelengths separately; these images arepassed to the image processing system 232 for processing. Imageprocessing at the first and third wavelengths is as heretofore describedwith reference to FIG. 22 ; however absorbance and scattering parametersare refined as indicated in step 1006. Image processing at the secondwavelength is performed by constructing the voxel-based model ofscattering and absorbance 1002 at the second wavelength, but withoutfluorescence parameter. The absorbance parameters at each voxel are thenratioed to provide a measure of heme oxygenation at each voxel, andsummed to provide a measure of heme concentration at each voxel.

The fluorescence, heme concentration, and heme oxygenation parameters ateach voxel are then mapped into tomographic image planes and displayedas tomographic images for inspection by the surgeon. The surgeon canthen use these images to distinguish tumor from adjacent and nearbynormal organ stroma. This embodiment is useful during surgery forvisualizing below-surface heme concentrations as well as fluorescenceduring a wide variety of surgical procedures; and is of particularutility for distinguishing malignant breast tissue from adjacent normalbreast tissue. In addition to cancer surgery, this embodiment is alsouseful for intraoperative identification of berry aneurisms,arteriovenous malformations, hematomas, and blood vessels.

An embodiment as heretofore described with reference to FIG. 11 iscapable of imaging heme and heme oxygenation below organ surfaces inreal time, and is capable of generating an additional tomographic imageindicating changes in heme concentration and/or heme oxygenation. Theseimages indicating changes in heme concentration and oxygenation areuseful for functional neuroimaging to confirm identification of foci ofseizure activity because neural activity depletes oxygen and increasesblood flow. Precise identification of foci of seizure activity is ofimportance during surgical procedures intended to alleviate epilepsy,and such surgery is occasionally done under local anesthesia to permitidentification of such foci. Since the apparatus can identify changes inheme concentration and oxygenation over a centimeter below a surface,this embodiment may also be used for functional neuroimaging through theintact skull, although resolution is less than that available with theskull open.

In an alternative embodiment, fluorescence excitation of fluorophores intissue is performed using a broad-beam source in the NIR at theexcitation wavelength, λ_(ex). It has been found that fluorescent lightemitted by concentrations of fluorophores like protoporphyrin IX is morestrongly absorbed by tissue in the 650-670 nanometer wavelength rangethan in longer wavelengths such as near 700-710 nanometers, and thatwith proper stimulus radiation, protoporphyrin IX will emit radiationhaving wavelengths from 650 to 720 nanometers. Similar effects may alsooccur with fluorescent light emitted by other fluorophores at similar orother wavelengths. This effect is due to the presence in tissue ofchromophores, substances—including hemoglobin—in tissue that absorb orscatter light more intensely at some wavelengths than others. Typically,though, while fluorescent radiation is emitted across this range, theemitted radiation is not uniform in intensity across the wavelengthband.

FIG. 14 illustrates spectra of fluorescent radiation emitted byfluorophores at several depths in tissue as observed above the tissue.These spectra have been normalized to the same peak level. A firstspectrum is that of fluorophores at the surface of the tissue, and isclose to the spectra of light as emitted by the fluorophores themselves.A second spectrum is that of light from fluorophores four millimetersbelow the surface as seen from above the surface, a third spectrum isthat of light from fluorophores one centimeter below the surface as seenfrom above the surface, and a fourth spectrum is that of light fromfluorophores two centimeters below the surface as observed from abovethe surface.

In an embodiment for use with the prodrug 5-ALA and the fluorophoreprotoporphyrin-IX, emitted fluorescence signals are measured at twowavelengths, λ_(1,2), each of which is longer than the excitationwavelength λ_(ex), using band-pass filters and a cooled CCD camera; inother embodiments emitted fluorescence signals are measured at multiple,meaning three or more, wavelengths using appropriate band-pass filtershaving center pass-band wavelengths at different wavelengths in therange of 650-720 nanometers. For example, if tissue has absorption andscattering characteristics similar to the tissue assumed in FIG. 27 ,and fluorescent light intensity I1 is measured through a filter havingbandpass 1610 between 680 and 690 nanometers while a fluorescent lightintensity I2 is measured through a filter having bandpass 1612 between700 and 710, a ratio of I1 to I2 will be considerably smaller when thefluorophore is two centimeters below the surface than when thefluorophore is at the surface of the tissue.

Embodiments for use with other fluorophores than protoporphyrin IX willrequire different calibration tables and may operate at wavelengthsother than those specified in the previous paragraph. Further, it isknown that optical absorption and scattering characteristics differ fromone type of tissue to another, and may also change somewhat with time asa subject's oxygenation levels change during surgery. In order toprovide for this variability, an embodiment, has a library with normaloptical characteristics of a variety of tissue types that thefluorescent tomographic microscope is expected to encounter duringsurgery. For example, the library may contain optical characteristicsfor normal brain grey matter and for normal brain white matter when themicroscope is used during removal of brain tumors such as gliomas. Anoperator then selects the appropriate tissue type for tissue in thefield of view, and optical characteristics of that tissue type are thenused for computation of fluorophore depth and distribution.

In an alternative embodiment, library has optical characteristics of avariety of tissue types with each tissue type measured under differentlevels of oxygenation. In these embodiments, the operator measurestissue oxygenation with direct or optical techniques and opticalcharacteristics of the tissue are determined based on both tissueoxygenation and tissue type.

Assuming that tissue absorption due to the fluorophore concentrations oftumor is much smaller than absorption by other chromophores of tissue,the light signals at each emission wavelength can be modeled using theexpression

$\begin{matrix}{{{\psi^{em}\left( {\overset{r}{R},\lambda^{ex},\lambda} \right)} \approx {Q_{F}{ɛ_{F}^{em}(\lambda)}{\int_{\Omega}{d^{3}r\;{\psi^{ex}\left( \overset{r}{r} \right)}{C_{F}\left( \overset{r}{r} \right)}{G^{em}\left( {\overset{r}{R},\overset{r}{r},\delta^{\lambda},D^{\lambda}} \right)}}}}}\ } & \left( {{Eq}.\mspace{14mu} 8} \right)\end{matrix}$where R^(→) is the vector corresponding to the detection point on thetissue surface, which is mapped onto a pixel or sub-ensemble of pixelson the CCD chip. Q_(F) is the quantum yield, ε_(F) ^(em)(λ) the emissionspectrum and C_(F)(r^(→)) the concentration of fluorescent molecules atlocation r^(→), where r^(→) is an arbitrary vector corresponding to thelocation of a point inside the interrogated tissue. In Eq. (8), lighttransport is modeled as a diffusive process. The fluence function ψ^(ex)is the excitation light field, and G^(em) is the diffusion equationGreen's function, which corresponds to the radiant exposure in responseto a light impulse at r^(→). For boundary conditions associated with aninfinite homogenous medium, it is given by

$\begin{matrix}{{{G_{\infty}^{em}\left( {\overset{r}{R},\overset{r}{r},\delta^{\lambda},D^{\lambda}} \right)} = \frac{\exp\left( {{- {{\overset{r}{R} - \overset{r}{r}}}}/\delta^{\lambda}} \right)}{4\pi\; D^{\lambda}{{\overset{r}{R} - \overset{r}{r}}}}},} & \left( {{Eq}.\mspace{20mu} 9} \right)\end{matrix}$where the diffusion constant is D^(λ)=⅓(μ_(a) ^(λ)+μ′_(s) ^(λ)) and thepenetration depth is δ^(λ)=√{square root over (D^(λ)/μ_(a) ^(λ))}. μ_(a)^(λ) is the absorption coefficient of the tissue while μ′_(s) ^(k) isthe reduced scattering coefficient.

A closed form expression is derived from Eq. (8) and Eq. (9), from whicha depth value can be estimated for each point imaged on the tissuesurface, using an approximation that all fluorescence emitted from thesurface is from a point-like distribution of fluorophore with molarconcentration C_(F) location at position r_(s) ^(→). This is equivalentto setting

${C_{F}\left( \overset{r}{r} \right)} = {C_{F}{\delta^{(3)}\left( {\overset{r}{r} - {\overset{r}{r}}_{s}} \right)}}$in Eq. (8), which then takes the form

$\begin{matrix}{{\psi^{em}\left( {\overset{r}{R},\lambda^{ex},\lambda} \right)} \approx {C_{F}Q_{F}{ɛ_{F}^{em}(\lambda)}{\psi^{em}\left( {\overset{r}{r}}_{s} \right)}{G^{em}\left( {{{\overset{r}{R} - {\overset{r}{r}}_{s}}},\delta^{\lambda},D^{\lambda}} \right)}}} & \left( {{Eq}.\mspace{14mu} 10} \right)\end{matrix}$where the depth of the distribution is

${{\overset{r}{R} - {\overset{r}{r}}_{s}}} = d$assuming that the detection point R^(→) is located directly above thesource of fluorescence at r_(s).

To illustrate how fluorescence spectra are affected by varying the depthof the fluorophore distribution, see FIG. 27 for protoporphyrin-IXspectra computed with Eq. (10) for depths varying up to d=20 mm for ananticipated typical absorption spectrum labeled and a constant value ofreduced scattering (μ_(s)′=1 mm⁻¹). As the object gets deeper in thetissue, the optical properties of the tissue demonstrate an increasinginfluence on shaping the observed spectrum because of the increasinglightpaths traversed by light. The information contained in thedistorted spectra may be distilled into a single quantity by calculatingthe ratio of the signal at two emission wavelengths,

$\begin{matrix}{\Gamma = {{\frac{\psi^{em}\left( {\overset{r}{R},\lambda^{ex},\lambda_{1}} \right)}{\psi^{em}\left( {\overset{r}{R},\lambda^{ex},\lambda_{2}} \right)} \times \frac{ɛ_{F}^{em}\left( \lambda_{2} \right)}{ɛ_{F}^{em}\left( \lambda_{1} \right)}} = \frac{{G^{em}\left( {\overset{r}{R},{\overset{r}{r}}_{s},\delta^{\lambda_{1}},D^{\lambda_{1}}} \right)}\ }{G^{em}\left( {\overset{r}{R},{\overset{r}{r}}_{s},\delta^{\lambda_{2}},D^{\lambda_{2}}} \right)}}} & \left( {{Eq}.\mspace{14mu} 11} \right)\end{matrix}$

where the intensity values at each wavelength are normalized with therelative signal strength of an undistorted emission spectrum, implyingthat Γ should be equal to 1 for d=0 mm.

In Eq. (11), Γ is independent of the diffuse excitation field. Thismeans that, in the point-source approximation limit, the ratio isindependent of scattering and absorption of the stimulus light.

Inserting Eq. (9) into Eq. (11), we find that the logarithm of thefluorescence ratio for an infinite medium is linearly related to thedepth in tissue of the fluorophores with a slope equal to the differencein penetration depth between wavelengths λ₁ and λ₂, This linearrelationship can potentially be used in estimating depth from a simplemeasured ratio in the case of diffusive medium of infinite spatialextent. A more realistic model for epi-illumination imaging may also beobtained assuming there is an index of refraction mismatch boundarybetween tissue and air. The relationship obtained is then similar—up toa multiplicative factor—for depth values larger than approximately twomillimeters. For smaller values than two millimeters, the relationshipis non-linear.

An alternative embodiment superficially resembling that of FIG. 11 asheretofore described constructs a fluorescence image cube having imagestaken at a different center wavelength in the range of wavelengthsemitted by a typical fluorescent material such as protoporphyrin IX,such as in the range 650-720 nanometers.

As described above with reference to Equations (8-11), the spectra offluorescent light passing through tissue is altered by absorption andscattering in the tissue, such that spectra of the received fluorescentlight encodes depth information. In an embodiment, two bandpass elementsare used to derive a spectral deformation and depth for the fluorescentlight captured in each pixel; in an alternative embodiment, three ormore bandpass elements are used to more precisely estimate spectraldeformation due to passage of emitted light through tissue and therebyestimate depth for each pixel of the image.

In an embodiment, the depth information determined as described above,using modeling of incident laser light and depth information derivedfrom spectral information in received fluorescent light, are used toprovide a refined depth information, and to thereby provide a refinedmodel of the tumor. In an alternative mode of operating this embodiment,the scanning laser may be disabled and depth information derived solelyfrom spectral deformation of fluorescent light stimulated by anappropriate flood illuminator.

In an embodiment using two image planes from the image cube in thewavelength range of fluorescence in the fluorophores in tumor, a twodimensional table is used. This table is generated according to theoptical properties of the tissue at the wavelengths of the bandpassfilters as determined by measurement through imaging of scattering of abeam, or from a library of optical properties as previously discussedherein. This table is indexed by intensity at the higher wavelengthbandpass filter in X, and intensity in the lower wavelength bandpassfilter in Y, the table having entries of approximate depth of tumorbased upon differences between intensity of fluorescent light passingthe higher wavelength filter and the lower wavelength filter. The tableis precomputed and stored in memory of the image processor to permitfast access. Intensity from each pixel in high wavelength and lowwavelength images are used with a table interpolation algorithm as knownin the art of computing to derive a depth associated with each pixel,and a three-dimensional model of fluorophore distribution is computed.Information from the model of fluorophore distribution is then displayedto the surgeon; this may be in the form of tomographic images, atopographic map, or by color coding of fluorescence magnitude withdetermined depth.

In another embodiment, the optical properties of the tissue asdetermined above at the wavelengths of the bandpass filter elements isused to compute the ‘slope’ for the log-linear relationship of the ratioof intensity at two wavelengths as described above with reference tosubstitution of Eq. (9) into Eq. (11). This ‘slope’ allows us to relatethe depth in tissue of the fluorophores to the difference in intensitybetween wavelengths λ₁ and λ₂. Then, the depth can be estimated based onthe measured ratio:

$\begin{matrix}{{d = \frac{{\ln\;\Gamma} - {\ln\frac{D^{\lambda_{2}}}{D^{\lambda_{1}}}}}{- \left\lbrack {\frac{1}{\delta^{\lambda_{1}}} - \frac{1}{\delta^{\lambda_{2}}}} \right\rbrack}},} & \left( {{Eq}.\mspace{14mu} 12} \right)\end{matrix}$

Alternatively the more general expression associated with asemi-infinite diffusive medium may be used, as described above withreference to Eq. (8-10).

Hyperspectral Imaging System Look-Up Table (LUT) Approach for Estimationof Tissue Optical Properties

The hyperspectral imaging system with white-light illumination can beused to estimate the tissue optical properties across the full field ofview to create maps of tissue optical properties, such as absorption(μa) and reduced scattering (μs′) coefficients. An embodiment of thismethod uses a look-up table (LUT), which has of a set of basis functionsof spectrally resolved diffuse reflectance including reflectance (Rd(λ))at known absorption (μa(λ)) and scattering (μs′(λ)). In a particularembodiment, the table is indexed by at least reflectance Rd, andwavelength λ, the components of reflectance spectra; and containsmultiple spectra, each captured at different values of known absorptionand scattering. The table has values recorded therein for absorption μa,and scattering μs′ at each table spectra. Captured spectra measured ateach pixel, which includes reflectance at a number of differentwavelengths, are fit to determine at least the closest table entries,and absorption and scattering parameters are read from the table. Inorder to conserve table space while providing adequate accuracy, lookupsinto the table for absorption and scattering parameters are interpolatedusing an interpolation method selected from the group of linear, cubic,or spline interpolation functions.

First, diffuse reflectance spectra, Rd(λ), are measured in phantoms ofknown and varying optical absorption, pa, and reduced scattering, μs′,across a range of values typically found in human tissues, includingboth normal tissues and pathological tissues such as tumor. Thecollected Rd(λ) at known μa(λ) and μs′(λ) are used to create a4-dimensional set of basis functions [Rd, λ, μa, μs′]. Wide-fieldspectrally-resolved reflectance images of tissue of unknown tissueoptical properties, e.g., acquired during surgery, are then decomposedinto Rd at each pixel. In an alternative embodiment, regularizedminimization is applied to fit the basis functions to observed Rd(λ) ateach pixel such that a best fit to a weighted sum of the basis functionsmatches Rd. The fitted basis functions yield estimates of μa(λ) and μs′(λ) at every pixel.

In particular embodiments, the estimated absorption and reducedscattering coefficients, μa(λ) and μs′(λ) are also used to determineadditional quantitative biomarkers that include but are not limited tohemoglobin concentration and oxygen saturation.

This technique enables a hyperspectral imaging system to collect diffusereflectance data, and apply a minimization algorithm unto a set of knownbasis functions, (in a particular example using the lookup tabledescribed above) to make explicit estimates of tissue optical propertiesacross the full field of view.

Hyperspectral Quantitative Fluorescence Imaging (qFI) Using a Look-UpTable (LUT) for Estimation of Tissue Optical Properties and a LightTransport Model

The hyperspectral imaging system with white-light illumination andfluorescence excitation light (e.g., light at 405 nm for PpIX) can beused to perform quantitative fluorescence imaging using a LUT and lighttransport model approach. In this embodiment, the estimates of μa(λ) andμs′(λ) determined by fitting spectra to entries of the table, andlooking up μa(λ) and μs′(λ) in the lookup table at every pixel m areused to create a ‘correction image’. The ‘correction image’ is then usedto correct the measured raw fluorescence based on a light transportmodel, such as the light transport model of Eq. 13 below, to producecorrected quantitative fluorescence images. This light transport modelof Eq. 13 is derived from a model published as Kim, A., et al.,Quantification of in vivo fluorescence decoupled from the effects oftissue optical properties using fiber-optic spectroscopy measurements, JBiomed Opt 15, 067006 (2010), the disclosure of which is incorporatedherein by reference.

$\begin{matrix}{f_{x,m} = {\left( \frac{\mu_{a,x}}{1 - R_{t,x}} \right)\left( \frac{F_{x,m}}{R_{m}} \right)}} & \left( {{Eq}.\mspace{14mu} 13} \right) \\{R_{t,x} = \frac{a_{x}^{\prime}}{1 + {2{\kappa\left( {1 - a_{x}^{\prime}} \right)}} + {\left\lbrack {1 + \left( {2{\kappa/3}} \right)} \right\rbrack\sqrt{3\left( {1 - a_{x}^{\prime}} \right)}}}} & \left( {{Eq}.\mspace{14mu} 14} \right)\end{matrix}$

-   -   where    -   R_(t,x) is the total diffuse reflectance. This parameter depends        on the internal reflectance parameter κ and the reduced albedo,        a′_(x), which in turn are defined as:

$\begin{matrix}{\kappa = \frac{\left( {1 + r_{id}} \right)}{\left( {1 - r_{id}} \right)}} & \left( {{Eq}.\mspace{14mu} 15} \right) \\{a_{x}^{\prime} = \frac{\mu_{s,x}^{\prime}}{\mu_{a,x} + \mu_{s,x}^{\prime}}} & \left( {{Eq}.\mspace{14mu} 16} \right)\end{matrix}$

-   -   With    -   r_(id)=−1.44n_(rel)−2+0.71n_(rel)−1+0.67+0.0636n_(rel), where        n_(rel)=n_(tissue)/n_(external) for matching internal and        external refractive indexes κ=1.    -   μ_(a,x) is the absorption coefficient at the excitation        wavelength (e.g., if using PpIX as the fluorophore, λ=405 nm);    -   F_(x,m) is the raw (uncorrected) fluorescence emissions; and    -   R_(m) is the diffuse reflectance at the emission wavelength.

Spectral unmixing (e.g., least squares approach) is then used toseparate the contributions from the fluorophore(s) being imaged, whichmay include PpIX and tissue autofluorescence, to construct a full-fieldof view image of fluorophore(s) concentrations, such as a PpIXconcentration map. This method enables quantitative fluorescence imagingqFI by correcting the raw fluorescence using explicit estimates of thetissue optical properties at the fluorescence wavelength and stimuluswavelength to create a ‘correction image’ to apply a light transportmodel of quantitative fluorescence. The fluorescence model presentedhere assumes that the optical absorption at the excitation wavelength hxis high relative to that at the emission wavelength, and as such, mostfluorophore absorption occurs close to the excitation light source. Themodel assumes that fluorescence migration paths are approximate by thoseof the reflectance photons at the emission wavelength.

In an embodiment, FIG. 15 is a flow diagram of a method of imaging.

FIG. 16 is a flowchart of obtaining and correcting a fluorescence imagefor absorption and scattering in a target, such as tissue, in order toquantify fluorescence and reflectance spectrally-derived images of thetissue. With reference to FIGS. 15 and 16 , and FIG. 11 , white light isprovided 502 to the target, or tissue, 152 by illuminators 166; andlight from the target is imaged 504 by camera through a tunable opticalfilter of the target while tuning the filter to a plurality ofwavelengths, the series of images forming a spectral image; providing astimulus light to the target 506 using illuminators 168; and taking afluorescence emission image of the target 508 through filters tuned toblock stimulus light. In an optional step 509 the fluorescence emissionimages of target 508 are captured as fluorescence spectra by sweepingthe tunable optical filter and capturing multiple images of the target,the fluorescence spectra are then decomposed 509 to provide individualfluorescence images related to each of several fluorophores expected tobe present in the target.

The images are then processed to determine absorption and scatteringspectral parameters; in an embodiment determining the absorption andscattering parameters is performed by interpolating in the table aspreviously described. The fluorescence emission image or images are thencorrected 510 by using the absorption and scattering parameters toproduce corrected fluorescence emission images giving a quantitativedistribution of each fluorophore, such as PPIX 524 and otherfluorophores 526 that may be native to the tissue or administered to apatient.

The absorption and scattering parameter spectra at each pixel are thendecomposed 512 to provide tissue component images mapping concentrationsof typical tissue constituents such as oxygenated and deoxygenatedhemoglobin; these are then added and ratioed 514 to provide a totalhemoglobin image 520 and a percent-oxygenation 522 map or image ofhemoglobin oxygenation in the tissue. The images, including corrected,quantitative, fluorescence, hemoglobin, and percent oxygenation, arethen displayed 516 for use in diagnosis.

Combinations

Various combinations of the elements may occur in a practical system.For example, in embodiments, optical system may be a surgicalmicroscope, while in other embodiments other optical systems having theability to illuminate with stimulus or white light, and to image inwhite or stimulus light and then in fluorescence emitted light may beused. In particular embodiments, the following combinations of featuresare anticipated:

An imaging system designated A, having an illumination device forilluminating a target, the illumination device adapted to selectivelyemit a white light and a fluorescent stimulus light; a device forreceiving light from the target, the device for receiving lightcomprising at least one optical output port at which at least a portionof the received light is provided as an output; a tunable filter forreceiving the portion of the received light provided as the output fromthe device for receiving light, the tunable filter being tunable to passa filtered portion of the received light, the filtered portion of thereceived light having a plurality of wavelengths selected by the tunablefilter; a high-resolution, broad-bandwidth electronic camera forreceiving the light of a plurality of wavelengths selected by thetunable filter and encoding a plurality of electronic images therefrom,the electronic images representing spectra at pixels of the images; anda processor configured to process the plurality of electronic images toform an image of the target; wherein the image includes an image offluorescent emissions by any fluorophore in the target.

It is understood that the system described in A is adapted for use witha target of human or other mammalian tissue.

An imaging system designated AB including the imaging system designatedA wherein the device for receiving light is a surgical microscope.

An imaging system designated AC including the imaging system designatedA or AB, wherein at least some of the electronic images the processor isconfigured to process are generated from fluorescence light received bythe surgical microscope and wherein at least some of the electronicimages processed by the processor are generated from reflectance lightreceived by the surgical microscope.

An imaging system designated AD including the imaging system designatedA, AB, or AC, wherein, in forming the image of the target, the processoris configured to use the electronic images generated from reflectancelight to compensate the electronic images from the fluorescence lightfor absorption and scattering while generating the image of fluorescentemissions from the target.

An imaging system designated AE including the imaging system designatedAD wherein the processor is configured to determine absorption and ascattering parameter at each pixel while compensating the electronicimages from the fluorescence light for absorption and scattering.

An imaging system designated AF including the imaging system designatedAE wherein a table interpolation is used to determine an opticalabsorption and a scattering parameter at each pixel.

An imaging system designated AG including the imaging system designatedA, AB, AC, AD, AE, or AF, wherein the tunable filter is a liquid crystaltunable filter.

The system designated AG or AH wherein a fluorophore in the targetcomprises protoporphyrin IX.

An imaging system designated AH including the imaging system designatedA, AB, AC, AD, AE, or AF wherein the tunable filter is an acousto-optictunable filter.

A surgical microscope system designated B, including: an illuminationdevice for illuminating a target, the illumination device adapted forselectively emitting a fluorescent stimulus light and a white light; atleast one optical output port at which at least a portion of receivedlight received from the target is provided; a tunable filter forreceiving the portion of the received light, the tunable filter beingtunable to pass a filtered portion of the received light, the filteredportion of the received light having a plurality of wavelengths selectedby the tunable filter and provided as output from the tunable filter; anelectronic camera for receiving the light of a plurality of wavelengthsselected by the tunable filter, the electronic camera adapted to convertthe light selected by the tunable filter to a plurality of electronicimages while the tunable filter is tuned; and a processor configured toprocess the plurality of electronic images to form images of the target,wherein the images of the target comprise images of fluorophores in thetarget.

A surgical microscope system designated BA including the systemdesignated B, wherein the electronic images the processor is configuredto process include images generated from fluorescence light receivedfrom the target and images generated from reflectance light receivedfrom the target.

A surgical microscope system designated BB including the systemdesignated B or BA, wherein, in forming the images of the target, theprocessor uses the images generated from reflectance light to compensatethe images generated from the fluorescence light for absorption andscattering in the target.

A surgical microscope system designated BC including the systemdesignated B, BA, or BB, wherein the tunable filter is selected from thegroup consisting of a liquid crystal tunable filter and an acousto-optictunable filter.

A method of imaging designated C including providing a white light to atarget; taking a series of images through a tunable optical filter ofthe target while tuning the filter to a plurality of wavelengths, theseries of images forming a spectral image; providing a stimulus light tothe target; taking a fluorescence emission image of the target;processing the spectral image to determine an absorption and ascattering parameter at pixels of the images; and correcting thefluorescence emission image using the absorption and scatteringparameters to produce a corrected fluorescence emission image.

A method designated CA including the method designated C furtherincluding decomposing the spectral image to provide images of componentsof the target, the components including specific scattering andabsorbing substances expected to be in the target.

A method designated CB including the method designated CA wherein thetarget comprises tissue, and wherein the components of the targetinclude oxygenated and deoxygenated hemoglobin.

The method designated CB wherein the images of components of the targetare combined to form images of total hemoglobin and hemoglobin oxygensaturation.

A method of imaging designated D including providing a white light to atarget; taking a series of images through a tunable optical filter ofthe target while tuning the filter to a plurality of wavelengths, theseries of images forming a spectral image; processing the spectral imageto determine an absorption and a scattering parameter at pixels of theimages; and decomposing the spectral image to provide images ofcomponents of the target, the components including specific scatteringand absorbing substances expected to be in the target.

A method designated DA including the method designated D wherein thetarget comprises tissue, and wherein the components of the targetinclude oxygenated and deoxygenated hemoglobin.

The method of claim DA wherein the images of components of the targetare combined to form images of total hemoglobin and hemoglobin oxygensaturation.

While the invention has been particularly shown and described withreference to a preferred embodiment thereof, it will be understood bythose skilled in the art that various other changes in the form anddetails may be made without departing from the spirit and scope of theinvention. It is to be understood that various changes may be made inadapting the invention to different embodiments without departing fromthe broader inventive concepts disclosed herein and comprehended by theclaims that follow.

What is claimed is:
 1. A hyperspectral imaging method for determiningdepth and concentration of fluorophores in tissue of a patient,comprising: capturing a white-light hyperspectral image cube of thetissue as illuminated with white light, the capturing performed using ahyperspectral imaging device that passes light from the tissue through atunable filter into a broad-spectrum electronic camera; capturing, usingthe hyperspectral imaging device, a fluorescence-light hyperspectralimage cube of the tissue as illuminated with fluorescence stimuluslight; analyzing spectral bands of the white-light hyperspectral imagecube near a wavelength of the fluorescence stimulus light and awavelength of associated fluorescence emissions to extract scatteringand absorption parameters of the tissue at the wavelengths of thefluorescence stimulus light and the associated fluorescence emissions;normalizing at least one selected pair of images of thefluorescence-light hyperspectral image cube, the selected pair of imagesbeing associated with two different respective wavelengths near a peakwavelength of the fluorescence emission; deriving, based upon thescattering and absorption parameters, the depth of fluorophores in thetissue at pixels from a ratio between the two images of the at least oneselected pair of images of the fluorescence-light hyperspectral imagecube, as normalized; and determining, using a voxel-based model and theextracted scattering and absorption parameters derived from thewhite-light hyperspectral image cube, and the depth of the fluorophoresin the tissue at the pixels and at least one image of thefluorescence-light hyperspectral image cube, fluorophore concentrationat voxels of the voxel-based model; wherein the step of derivingcomprises comparing the ratio between the two images of the at least oneselected pair of images to a model of depth-dependent spectralproperties of the fluorescence of the two images of the at least oneselected pair of images.
 2. The hyperspectral imaging method of claim 1,further comprising, prior to the step of comparing, determiningparameters of the model of the depth-dependent spectral properties fromthe scattering and absorption parameters.
 3. The hyperspectral imagingmethod of claim 1, the model of the depth-dependent spectral propertiesspecifying the depth as d=[1n(Γ)−ln(D₁/D₂)]/[1/δ₂−1/δ₁], wherein (a) D₁and D₂ are diffusion constants for the two different wavelengths,respectively, (b) δ₁ and δ₂ are penetration depths for the two differentwavelengths, respectively, and (c) Γ is the ratio, the hyperspectralimaging method further comprising determining each of D₁, D₂, δ₁, and δ₂from the scattering and absorption parameters.
 4. A hyperspectralimaging method for determining depth and concentration of fluorophoresin tissue of a patient, comprising: capturing a white-lighthyperspectral image cube of the tissue as illuminated with white light,the capturing performed using a hyperspectral imaging device that passeslight from the tissue through a tunable filter into an electroniccamera; capturing, using the hyperspectral imaging device, afluorescence-light hyperspectral image cube of the tissue as illuminatedwith fluorescence stimulus light; analyzing spectral bands of thewhite-light hyperspectral image cube near a wavelength of thefluorescence stimulus light and a wavelength of associated fluorescenceemissions to extract scattering and absorption parameters of the tissueat the wavelengths of the fluorescence stimulus light and the associatedfluorescence emissions; normalizing at least one selected pair of imagesof the fluorescence-light hyperspectral image cube, the selected pair ofimages being associated with two different respective wavelengths near apeak wavelength of the fluorescence emission; deriving, based upon thescattering and absorption parameters, depth of fluorophores in thetissue at pixels of the selected pair of images from a ratio between thetwo images of the at least one selected pair of images of thefluorescence-light hyperspectral image cube, as normalized; anddetermining, using a voxel-based model and the extracted scattering andabsorption parameters derived from the white-light hyperspectral imagecube, and the depth of the fluorophores in the tissue at the pixels andat least one image of the fluorescence-light hyperspectral image cube,fluorophore concentration at voxels of the voxel-based model.
 5. Thehyperspectral imaging method of claim 4, comprising: in the step ofnormalizing, selecting a plurality of pairs of images of thefluorescence-light hyperspectral image cube; and in the step ofderiving, collectively evaluating a plurality of ratios, each obtainedfrom a respective one of the plurality of pairs.
 6. The hyperspectralimaging method of claim 5, the step of collectively evaluatingcomprising fitting, to the plurality of ratios, a model ofdepth-dependent spectral properties of fluorescence emissions, todetermine the depth of fluorophores from said fitting; and thehyperspectral imaging method further comprising, prior to the step offitting, determining parameters of the model of the depth-dependentspectral properties from the scattering and absorption parameters. 7.The hyperspectral imaging method of claim 4, the step of normalizingcomprising dividing intensity of pixels of images in the at least asubset of the fluorescence-light hyperspectral image cube by (a)intensity of pixels in an image of the white-light hyperspectral imagecube corresponding to wavelength of the fluorescence stimulus light and(b) intensity of pixels in an image of the white-light hyperspectralimage cube associated with same wavelength as the respective image inthe at least a subset of the fluorescence-light hyperspectral imagecube.
 8. The hyperspectral imaging method of claim 7, further comprisingcapturing the white-light hyperspectral image cube and thefluorescence-light hyperspectral image cube at same set of wavelengths,the step of normalizing comprising normalizing at least two pairs ofimages of the fluorescence-light hyperspectral image cube.
 9. The methodof claim 4 wherein the step of analyzing comprises setting up thevoxel-based model with the scattering and absorption parameters andrefining the scattering and absorption parameters.